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ABSTPACT 

Tvo instructional sethods for a freshsan physics 
course were coipared: audio-tutorial (AT) instruction in a learning 
center and lecture-recitation-laboratory (IRL) instruction. Both 
randoa assignment and student preference were used to distribute the 
575 students between the two sethods* Course content, hosework, and 
tests were identical. AchleYesent was evaluated by exas grades, 
adjusted for SAT aath scores and aath pretest scores. Questionnaires 
were used to gauge student attitudes toward the course. To detersine 
long range effects of the rwo instructional aethods, an analysis was 
Bade of the progress of the students, in teras of choice of physics 
classes, was designed to aeasure content and bnhavior under AT and 
IBl instruction, to show variations between the two sethods, and to 
detersine interrelationships aaong student achievesent, student 
evaluation of teachers, and teacher behavior. Besults Indicated that 
AT and LPt lead to sisilar levels of achievesent, but students with 
low sath aptitude did better under AT while high aath achievers did 
better under LPl. The observation instrusent is included. (.U) 
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I. Introduction 

This report is concerned with the evaluation of two methods 
of instruction used in a college physics course at Cornell University. 
The two methods are audio-tutorial instruction and lecture-recitation- 
laboratory instruction. The evaluation was conducted during the aca- 
demic year 1973-74. 

Thj research program investigated a number of related areas: 

1. The effects of the two methods of instruction on levels 
of student achievement and on student attitudes toward the course. 

2. Longer term effects of the two methoc'iS in terms of 
achievement and areas of interest subsequent to the course. 

3. Characteristics of the major components of the two me- 
thods of instruction, i.e., lecture, recitation, and learning center 
instruction. 

Following a description of the course, the methods of in- 
struction, and the methods of assigning students, these three areas 
will be considered in detail. The report concludes with guidelines 
for the use of audio-tutorial instruction and lecture-recitation- 
laboratory instruction. 

A number of persons at Cornell University have made signifi- 
cant contributions to this research. In particular, David Macklin of 
the Center for Improvement of Undergraduate Education has contributed 
to every phase of the work. Stephen Brock, also of the Center for 
Improvement of Undergraduate Education, served as project director. 
Walter Federer and Jason Millman provided advice on experimental de- 
sign and statistical analysis. Donald Dickason, Donald Burgett, and 
Barbara Hlrshfeld assisted by providing student aptitude test Informa- 
tion. Patricia Muslck, Catherine Clark, and William Love joy per- 
formed invaluable service as observers of instruction. Tlie Division 
of Basic Studies of the College of Engineering, and the faculty, 
teaching assistants, and students in Physics 112 cooperated with the 
research project despite the occasional inconvenience it entailed. 

- 1 - 
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The Course 

The course in question, P112, Mechanics and Heat, is a one-seroester 
freshman level physics course offered to approximately five hundred and 
fifty five engineering and physics majors each spring semester. In 1974 
97% of the students were freshmen, 91% were males, and 90% or more were 
enrolled in engineering. 

The subject matter of the course included classical mechanics, 
special relativity, and heat. The course textbook was Fundamentals 
of Physics by David Halliday and Robert Resnick. This text was supple- 
mented by notes on relativity written by the course professor. Jay Orear. 
The Methods of Instruction 

The course was offered to students via one cf two methods or 
treatments — a lecture-recitation- laboratory (i; anOard) method of 
Instruction, and an audio-tutorial (a.t.) method. The content of the 
instruction was similar in the two methods. In adoition, students in 
both had the same homework assignments, the same .ab assignments, and 
identical quizzes and examinations. Taerefore the difference in treat- 
ments was primarily one of instructional method, rather than of content, 
assignments, or examinations. 

The standard method of instruction included two hours of lecture 
and two hours of recitation per week, and a two-hour laboratory every 
other week. (Lecture and recitation are described in more detail in 
part IV S.) On the average, twenty six students were assigned to each 
recitation section and twelve to each lab section. Recitations and labor- 
atories were taught by faculty members and graduate teaching assistants. 

The audio-tutorial method Included one hour of recitation per 
week. This hour of group instruction was included primarily to provide 
an opportunity for group interaction, as well as student contact with 
one particular instructor and an opportunity for testing. In the audio- 
tutorial method, all other instruction took place at the student's 
convenience in a learning center. The learning center was staffed by 
tutors forty-seven hours per week. Materials available in the learning 
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cwiitec I'lciadtjJ »U-motvir.r,iuLuii .-'.jpiL-af.iu-i f«»i: :;.^ir-i'.Minousirations , 
laboratory equipment, audio-tape comnientacios , and slides, llie taped 
coounentar les and slides were coordinated with a study-guide developed 
by the author and Professor John Sllcox. Audio tape-slide-study guide 
sequences were Intended in particular to assist students in learning 
to solve physics problems. (Tlie students* usage of the learning center 
is described in Part IVA.) 
Methods of Assigning Students to Treatments 

The research involved an experimental design intended to answer 

questions related to: 

1. the methods of instruction (audio-tutorial or standard) 

2. the procedures of assigning students to methods of in- 
struction. 

The two methods have already been described. The two procedures 
of assigning individual students to these methods were (a) random assign 
ment, (b) assignment according to student preference. 

Thus our experimental design can be diagrammed as follows: 

Audio-tutorial Standard 

Random 



Preference 



In detail, the students were assigned as follows: 
Each enrolled student was assigned to one of twenty- two recitation 
sections. Students were assigned to sections at a given class hour on 
the basis of their other scheduled classes and their personal preferences. 
The fifteen sections scheduled at four of the class horrs were then se- 
lected for inclusion in the study. (These four class hours were selected 
because three or more sections were scheduled at each of these hours.) 
Eight of these sections became standard recitation sections. Seven be- 
came audio- tutorial recitation sections. 
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At two of tlw four class hours included In th^ study, .^5Cuddii!:f5 
were randomly as«ir,ned to treatments and to recitation sections within 
the treatments. At the other two hours, students were assigned to 
treatments on the basis of their preferences as expressed on a question- 
naire administered at class meetings during the first week of the 
semester. Within each of these two hours, students were then^randomly 
assigned to specific sections within the preferred treatment. In 197A 
the result of these procedures was the formation of four audio-tutorial 
random (ATIO sections, four standard random (STR) sections, three audio- 
tutorial preference (ATP) sections, and four standard preference (STP) 
sections, as indicated in the following diagram. 





Audio-tutorial 


S tandard 


Random 


4 


A 


Preference 


3 


A 



Eight teachers, including two faculty members and six graduate 
students, taught these fifteen sections. Seven teachers taught one 
audio-tutorial and one standard section each. One graduate student taught 
two standard sections, only one of which was included in the study. 

We turn now to consideration of the three areas of research re- 
ferred to in the Introduction. 



* At each of these hours, some students were assigned to a method 
contrary to their preference in order to balance class sizes. These 
students are not included in the subsequent analyses. 
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A. Student Achievement 

In consiaerlng the achicvemenL of the student:;, we Cv^^CL-!ptu:l L- 
ized the problem an one of posslblG traic-treatment Interactions. 'That 
is, we determined whether student characteristics such as aptitudes or 
previous achievement predicted greater achieveot'nt in one method (treat- 
ment) than in the other. The res'-Us of research completed early in 
this project indicated that in f such an interaction existed when the 
course was offered in the Spring semester, 1973. In that case, two 
"traits" were important achievement in math, and math aptitude. Stated 
briefly, the 1973 results were as follows. 

Students with very high math aptitude (SAT Math scores of 725 
or higher) and high math achievement on a course-specific pretest, had 
higher predicted grades in the standard method than did comparable stu- 
dents in the audio-tutorial method. Students with relatively low math 
aptitude (SAT Math scores of 625 or lower) and low math achievement, had 
higher predicted grades in the audio-tutorial method than did their 
counterparts in the standard method. Predicted grades of students hav- 
ing traits in the intermediate range of math aptitude and achievement 
did not differ significantly in the two methods. These research findings 
and the research design used in 1973 are described in detail in the ac- 
companying paper, "A Trait-Treatment Interaction in a College Physics 
Course" (Appendix A) . However, there are two points to be added to this 
report. The first is that math achievement was a more important trait 
in this research than was mathematical aptitude. That is, the math pre- 
test scores accounted for a greater proportion of the variance in the 
grades than did the SAT math levels. The second point is that within 
the audio- tutorial method there were differences in achievement between 
the students assigned randomly and those assigned according to their 
preference. When math aptitude and math achievement were taken into 
account, audio-tutorial students who had been assigned randomly had sig- 
nificantly higher achievement than those assigned according to their 
preference. This result is suggested in Table 3 of the Trait-Treatment 
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Interaction paper" and wa» verified by waans oi' a Johnson Nyyman ana- 
lysia of the interaction. 

Wien tlie conr^iC- va.; offered in the Spring KtMi'riSter 107A, 
sought to determine whether tht» results of tho previous year would be 
reproduced. To this ond, we did not reveal thcKe results to ai y of 
the cour5?c instructors other than the faculty inemoer who had directed 
the course in 1973. We also tried to maintain fairly similar conditions 
in the course by again having each teacher teach a section in both methods, 
by having uniform quizzes, homework and exams, and so forth. However, 
changes beyond our control did occur. The major ones were as follows: 
(1) a different faculty member directed the course and lectured to the 
"standard" students than in the previous year; (2) the two interim exama 
were one and a half hours long Instead of one hour; (3) the exains were 
constructed primarily by the course professor rather t-'-ian by committees 
of instructors; (4) the lecturers utilized an "Instant Response System" 
which allowed the professor to learn student responses to questions by 
means of push-buttons at the students* chairs; (5) furthermore, the size 
of the course was decreased somewhat and the range of student abilities 
was thereby diminished because more students of high math aptitude and 
achievement had been encouraged to take the course during the previous 
term. 

These changes are impori.ant in terms of comparing the research 
conducted in 1973 with that conducted in 1974. Taking each change in 
order: 

(1) The change in professor decreased the degree of coordina- 
tion between methods because the professor who gave the lectures in 1973 
was one of the developers of the audio- tutorial material and the newly 
assigned professor had no experience with the audio-tutorial method of 
Instruction. (This change may also have affected the difficulty level 
of the lectures. I believe that the professor in 1974 directed the lec- 
ture to students having lower ability levels than did the professor in 
1973.) 

* I am gratefu:* to Lee J. Cronbach for poLntinp. out this effect. 



Er|c 9 



(2) 'ilu* .U>ti,^«i .'.Miidtuu. Ion LL;v.t* im rLoJ p :ui)ai)1.y rw^sulcLid i.t 
d.>.-f»'.uiO!il I iiuo prerft.urt.' in th:? otairs in 1974 rcl.itivc to ]97"1. 

(*;) ])w' to Lh,' r.:.rl: tlKil Lhv c'.Mms w.-ro. pri!i!iri ] y con.? Vrurtfid 
by ouo ludLviaual In 1974 and by comm.Utc in 1973, tlu; cxarus In J07A 
apptiuc to h;\vo. been more honojient'oiii-; and to have neasuri'd a narrower 
r.inj^C! of student lc.irnln>; than did those in 1973. 

(4) The "instant response system" has been found by Professor 
R. LictauerK^t Cornell to lead to increased student interest and to 
maintain higher levels of student attendance at lectures. 

(5) The decreased range of student abilities affected both 
what was required of instructors in the classroom as well as the level 
of significance of statistical results. 

These differences have been included here to help point out the 
difficulties inherent in attempting to reproduce a study of teaching, 
and to put in perspective the results of the 197A research concerning 
achievement. 

In 1974, the research concerning student achievement indicated 
major differences between the effects of random and preference assignment 
procedures: 

(1) Among students who were randomly assigned to the methods 
there was a minor indication of an interaction similar to that found the 
previous year. Tliat is, students who had higher math achievement did 
slightly better in the standard method than in the audio-tutorial method. 
Those with lower math achievement did slightly better in the audio- 
tutorial method than in the standard. However the difference in the 
slopes of regression lines and therefore the interaction that was ob- 
tained were not statistically significant. Furthermore, math aptitude as 
measured by SAT math scores was not highly related to grade in either 
method for randomly assigned students. (Please see Figure 1.) 

1) R. Littauer, "Instructional Implications of a Low-Cost Electronic 
Student Response System," Educational Technology , XII (Oct., 1972), 69. 
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2 3 4 5.6 
Math Pretest (Number of Errors) 



Figure 1 
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In c»»ir.bi nation vitb the 1973 nvsnU:;, tbos.- rt^sulL-S indicates 
thai Uiorre may b- !\ I'UuUnxcy to a i ra i t-t i-oatiiuMvt i nterar t Lon for tbi> 
F-tandara .->.".! audiD-Lurorlal tn^'lboJ^ aiul tliat. UiLi Lt-udcncy m-\y ba r*- 
laforcecl to pc'i'itor or lesser de«ieps under different couroe conditions 
and wltb difftu-ont course in.itractorK . 

(2) ,u,oag students a^jsigaed by preference* those who se- 
lected tbe standard method clearly achieved higher final grades than 
did those who selected the audio- tutorial method. Using the Johnson- 
Neyman technique to detect regions of significant differences between 
treatments, we compared the predicted grades for the audio- tutorial 
preference and the standard preference students. We obtained the fol- 
lowing results: 

a. For the range of student traits in our sample, the 
predicted grade was higher in every case for the standard preference 
group. 

b. For a range of SAT Math values of about 630-670, the 
predicted grades were significantly higher in the standard preference 
group (level of significance = .05). (Note that these SAT math values 
are the actual scores received by the students, rather than levels as 

reported in 1973.) 

Tliese results are sketched in Figure 2. Statistics are given 
in Table 1. In Figure 2 the cr^ line is the line Indicating equal pre- 
dicted grades in the two treatments. Above this line, predicted grades 
are higher in the standard preference group. Below this line, predicted 
grades are higher in the a.t. preference group. Due to the fact that 
no students had SATM scores which fell below the a<r^ line, the standard 
preference treatment had higher predicted grades than the a.t. preference 
treatment for all students in the population. In addition, for an ellip- 
tical region bounded by SATM scores of 630 to 670, predicted grades were 
significantly higher in the standard preference treatment than in the 
a.t. preference treatment (level of significance » .05). Thus the 

* Note that the preference students who were assigned to a method con- 
trary to their preference are not Included in these and subsequent 
analyses. However such students were included in the 1973 analysis. 
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Regression 
Equations 



Troai manL s 

Audto-tuLorial Preference !>.andacd Preference 



52 



z = lA.283+.527x-.012y 



N^^ 88 



u = 16.201+.516x-.017y 



Means 



- 2.50 

- 689.6 
7 - 7.04 



- 2.68 
7^ = 674.0 
u = 6.23 



Standard 
Deviations 



^x' 



1.67 
54.6 
3.22 



» 1.71 
V =54.8 
= 3.19 



Correlation 
Coefficients 



xy' 



-.437 



xz » +.365 
^yz = -.329 



^xy" 



-.530 
'^xu = .430 
^yu = - . 436 



X = errors on math pretest 
y » math SAT score 

z =« final grade in the audio-tutorial preference treatment 
u ^ final grade in the standard preference treatment 



porfornanci^ of the slMiidard preforence studoars sutpas^ed that of the 
a.t. profrr.nco stti.h-ti! . Tt hi of intiorasf to note that th^T a.t. pro- 
ferotK-.o }:ttidcnt;3 rc^cei.vod f',radcr. Jn thfir other coiii-s:.'s which Wi'Tf 
lower than the grades achieved by each of the three otlier groups of 
students, (standard preference, a.t. random, and standard random). Thus 
lower performance in Pli:'. was not counterbalanced by hi^^her performance 
in other classes. 

Assuming that there were no systematic differoaceR between stu- 
dents given the random or preference assignment procedures, the dif- 
ference in course grades between a.t. preference and standard prefer- 
ence students and the lack of such differences within the randomly as- 
signed groups require an explanation. It apparently is to be found in 
terms of the motivation of the students who selected the a.t. method. 
First of all, what were these students like in relation to those who chose 
the standard method? 

From the Initial questionnaire we learn: 

Twelve of the fifty-two (23%) who chose a.t. and stayed with 
their choice had taken two or more years of high school physics, nine 
of the ninety (10%) who chose standard had also taken two or more years 
of high school physics. Fifty of the fifty-two (96%) who chose a.t. .said 
that they would succeed well or very well in a science course without 
pressures from teachers or other students. Only fifty-nine of the ninety 
(66%) who chose standard so stated. 

T\^ent3f-eigbt of the tifty-tyo (54%) who chose a.t- thought P112 
would be difficult, whereas sixty- three of the ninety (70%) who chose 
standard thought P112 would be difficult or very difficult. 

None of the students who chose a.t. was registered in Math 191, 
whereas eight standard students were registered in this math course. 
Math 191, a course taught in an "exam tutorial", self-paced format, is 
usually a prerequisite to enrollment in P112. Apparently students who 
were behind in mathematics and who may have faltered in the non-traditional 
mode of instruction in M191 did not choose the audio-tutorial method in 
P112. 
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In regard to thoLr previous .-:per U^ncc; in science and maCh 
<:.;urt;-;. o'.l.y vV<Ui ^.T t li - f i t." t y»{ '.v-o (1T>^0 v/ho chor.c a.t. thou-hl: Tl:- 
citation-. had been vmt/ lu-lpfnl in rUo. V-^t. In contrnst, thirty-nhui 
of tlKi niii^'ty standard itudcuUs thoujihc rocitatioas had bevn 

very helpful. 

As far as math achievement and aptitude are concerned, those 
who chose a.t. did slightly better on the math pretest (2.5 errors vs. 
2.7) and had higher average math SAT scores (690 vs. 674). Mean SAT 
math scores, math pretest scores, and final grades are given for each 

treatment group in Table 2. 

As far as students' experiences in P112 are concerned, on a final 

questionnaire students reported the following: 

Seven of the forty-five a.t. students responding (16%) said 
they stopped regularly attending the learning center by the fourth week 
of the course. None of the seventy-four responding students in the 
standard preference sections claimed to have stopped attending lecture 

that early in the semester. 

A.t. preference students appear to have spent fewer hours in 
the learning center than a.t. random students. Moreover, twenty-four 
of forty-four a.t. preference students (55%) said that they did not re- 
gularly attend the learning center. The proportion in the a.t. random 
groups was fourteen out of forty-four (32%). 

A.t. preference students said they spent about five hours 
per week outside class on P112. vs. 8.6 hours for standard preference 
students. 

As far as achievement is concerned: 

The standard preference students had a considerably higher 
average grade on the first prelim than did those who chose a.t. Other 
test grades did not differ significantly. 

Finally, as mentioned earlier, the a.t. preference students 
had slightly lower grade point averages in their other courses than 
standard preference students. 
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Tlu' picture which emev^^^ oi^ Liu- "prol.'.\-aoe" j-likUmu.-; 
Ih.' nuaio-luvorl.,1 ,n?t!unl u; that of a C.iirly ronfldont r.vonp of studcttn 
with a Cairly r.ocd pr..paririon In p!,ysicr.. Monnw.M- they apparently i.ad 
not touud foLual c L:u;.;roo'.,i 1n..Lructioti particularly t,..efur "ia ucicnc. ..nd 
tuath couruc.s in tho past. They selec ted the a.t. m^athod but in many csc-s 
mde very little »so oF the available materials. Perhaps they put too 
much faith in their preparation in physics, as indicated by the low scor«.s 
on the first test which mainly emphasized material treated in high school 
physics classes. 
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B. SLudoiit AttiLud^vs 

StuU'^^.t attitu.lrs to.Mvd H.f audio-tutorial imd -standard methods 
usvd In ril.2 .•valua.-.! V.y i.i.au-. <^l a mu.':^ i om.a i r.^ r.-Pn.T 1 < d by . 

c^rvidents nt their Jant roclLotLon meeting of the ncmcisLcr. This pvo- 
ceduL-e was followed in 1973 and in 1974. The quastlonnalrcs used ia 
1973 and in 1974 were similar in construction, involvinf; cvaluatio.ii-! 
of various aspects of the course and of the methods of instruction, 
as well as ratings of instructors. 

In order to determine student attitudes toward the methods of in- 
struction, two fairly global statements were included in each question- 
naire. Students were asked to indicate the extent of their agreement 
with each statement by indicating either strongly disagree, disagree, 
neither agree nor disagree, agree, or strongly agree. The statements 
are as follows (with the appropriate method indicated in each case) : 

1) In general, I have been satisfied with the audio-tutorial 
(standard) method of instruction used in PH2 

2) I am glad I took the a.t. (standard) version of P112 
rather than the standard (a.t.) version. 

Student responses to these two statements are reported in Table 3 

for both 1973 and 1974. 

In general students tended to indicate greater agreement with the 
second statement than the first statement. That is. some students indi- 
cated that they were not satisfied with the method they received, but 
that they nonetheless were glad they took that method rather than the 
other method. 

As far as assignment procedures are concerned, preference groups 
uniformly indicated greater agreement with both statements in both years 

than did random groups. 

As far as methods of instruction are concerned, audio-tutorial stu- 
dents Indicated greater agreement with the second statement in 1973 
than in 1974. Standard students' opinions did not shift significantly 
from 1973 to 1974. 



( >t lo-nai re St:nteir>ent« . * gf^j QQpY AVAILABLE 

Sli»."-'f>'ur 1. ill }',t:.v'ro.l, T h:tvo bt'^i'n aar.i;; f i (t^tl v/i.fh tli.e auctlo-ttifor.i 

(standard) in>,'thod of instruction used In Pll?.. 

Perc.entaK« o^ Responses 
Year Croup Total N Agree/Strongly Agree Neutral Dlsagroe/Strongly 

Disagree 



1973 


ATR 


51 




41 


16 


43 




ATP 


38 




53 


18 


29 




STR 






43 


31 


26 




STP 


73 




50 


25 


25 


1974 


ATR 


i L 
kk 




jU 




43 




ATP 








9 


34 




STR 


52 




35 


25 


40 




STP 


69 




55 


17 


28 


Statement 2. 


X am glad I 


took 


the a.t. 


(standard) version of 


P112 


rather 


than 


the standard 


(a.t. 


) version. 






1973 


ATR 


52 




48 


27 


25 




ATP 


37 




70 


11 


19 




STR 


93 




59 


26 


15 




STP 


74 




67 


23 


10 


1974 


ATR 


44 




39 


20 


41 




ATP 


44 




57 


14 


29 




STR 


53 




62 


28 


10 




STP 


69 




75 


16 


9 



* Only those students who had initially preferred the method they were 
assigned to were included in the totals for preference groups. Students 
assigned to preference groups contrary to their preference tended to be 
very negative about the methods they received. 
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Standird and and io-rntorial stiu'ent:* did not dlll'or in the pa^li'rn 
of Ihoii- i*•>^:nan :; to at.atoiu'ut: 1 eltlior in 1973 or iu Chaiv;:js 
v/hlch occutM^^l in Tl sooaso;; to slai ononl 1 cons ist ed in I rss n^./,rcv^» iirni: 
with the statement: ninonj^ randomly asBigned audio-tutorial and standard 
students in 197A compared V7ith 1973. 

Responses to statorr.eut 2 indicated fairly simiLar reiioonses auioa^j 
audio-tutorial and standard students in 1973. lloivfver there was de- 
creased agreement with the statement among audio^tutorial students in 
1974 relative to 1973 and slightly increased agreement among standard 
students in 1974 relative to 1973. 

In summary: 

1) Students assigned by preference expressed more positive atti- 
tudes toward the method they received than did students assigned by ran- 
dom procedures. 

2) Audio-tutorial and standard students tended to be equally 
satisfied with the method they used (Statement 1) . 

3) In 1974, but not in 1973, standard students were more likely 
than audio-tutorial students to say that they were glad to have taken 
their method of instruction (Statement 2). 

The greater disaffection with Instructional method among audio- 
tutorial students may have been due to problems experienced in 1974 in 
coordinating the two methods of instruction, to increased problems with 
the availability of laboratory apparatus, to personnel changes, or to 
other causes. 

In summary, therefore, we have learned that student achievement did 
not differ in 1974 among audio-tutorial random and standard random stu- 
dents* However, achievement was higher for standard preference students 
than for audio-tutorial preference students. In combination with the 
finding that in 1973 achievement was significantly higher for audio- 
tutorial random students than for audio-tutorial preference students, 
one has reason to question the advisability of allowing students to 
choose a method of instruction. 
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111 rv'i'.ai'fl to atrituJe., we fi)uiid tiiaC i;tuat:'n£,M aasiiitied l>y 

prcic't-rnc.' v.'-.-r..' rv>ro nat i.sfiod with the luvtho-l of InsLrncCion th.^y 
rcccilv^H thiin w.-rf r:'.u('ot-i ! y asr-i {'.H'"*'"! studcMits. Furt-hornoro , In 1974 
audLo-tut.-L ial i-.L-uioniii wore more likely to say Lliey would havo pru- 
forrv'd tlK! ;ilt;ernat:ivt» method than were otaudard students. 
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\i\ orJov to drli. Liiuuo v^iic'lber' there were Ion;;'/ r^ran<;e vSft^cir. of 
the dlfiort aces In tuetUodt; of iastrucLlon, we aiialy^^.od tlie proj^^ress of 
tho ea^jlncer ing students Lu the Vj7 ^ satPplo (ciboui o\ Uu- s.unpkO. We 

looked at the stiideatti In the a.t. and standard random sections (ATR 
and STR) ^ and those in the a.t. and standard preterencc. sections 
(ATP and STP) . As in the res^iarch reported in Appendix A, we included 
only those students in the random sections who had been originally 
assigned to the recitation which they continued in, and only those pre- 
ference students who actually obtained the method they preferred and 

and who did not transfer sections • Students who dropped P112 or trans- 
it 

f erred sections were excluded. We sojght to determine: 1) whether 
these groups of students enrolled in tho. same selection of engineering 
courses; 2) whether there were any negative effects of the audio-- 
tutorial method w^ -n students returned to a lecture-recitation* lab 
format in their science* courses; and 3) whether audio-tutorial and 
standard students had different attrition rates in later semesters* 

A word is in order about the typical pattern of courses for sopho- 
more engineering students at CornelK Usually these students take one 
physics course (P213) in the first semester and two (P214 and P216) in 
the second. They also take two engineering courses each semester, 
usually from a selection of spiventeen ''basic studies*' courses. We found 
however, that the approximately 2A0 engineering students in our sample 
who enrolled as sophomores enrolled in a total of 35 different engin*- 
eering courses » including some upper division courses • On the other 
hand, only six engineering courses had substantial numbers of these 
students enrolled. These six courses were: 

Computer Science 202 (Computers and Programming) 
Engineering 9160 (Introductory Probability) 
Engineering 4210 (Introduction to Electrical Systems) 
Engineering 1021 (Mechanics of Solids) 

^ See p. 6 of Appendix A. 
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iir/iij^t^L' LH/, (i/Ml (;•; orhaa ical rr<3;)c*r f: lc\s of MaLori a Is) 
Kn.^ in^»or i n;» (Introduct loa to TI]ormoc!yaru c ;) 

Torccnua^-a of tlu sindciits from each rosearch {;roap in each of Lheiio, 
courses arc iadicati^d In Table 4* It is apparent that thert^ vcre no 
drastic differences in enrollment patterns from the four researtU ;;.onn.i. 



Table 4. Enrollment in Engineering Courses 
(Percentage of Groups) 

Course ATR ATP STR STP 



Computers and Programming 29 33 33 30 

(202) 

Introductory Probability 73 50 66 61 

(9160) 

Introduction to Electrical Systems 63 70 58 58 

(4210) 

Mechanics of Solids 51 47 57 64 

(1021) 

Mechanical Properties of Materials 25 20 31 24 

(6261) 

Introduction to Thermodynamics 29 33 22 2}J 

(3631) 

Numbers in groups 51 30 86 74 

Bi*Fore turning to the achievement of the engineering students in 
engineering and physics courses^ four poi .ts should be noted: 

1) The STR group Included 96 engineering students in Spring 
1973^ and the ATR group included 57. This standard random grovp had 
somewhat higher SAT math scores and somewhat lower math achievement 
scores than the a.t. random group. As attrition occurred during the 
next two semes terSt these relationships were maintained. 

2) The ATP group included 37 engineering students In Spring 
1973 and the STP group included 83. This standard group had lower SAT 
math and math achievement scores than the- a.t. preference group. As 
attrition occurred, the math achievement and the SAT math differences 
continued through Spring 1974 although the differences became attenuated. 
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uiricrinl'Ly lower aVw-i:.i,Vi^* ^'tau.^s in Vi^l limii d\d rho''»* i;\ ihe a.t. 
ii!OthoiU Tlu' aiulio-tiitor ial (Uo fVncncr^ un^Jnverin;; st:iKU:aLs lind re?- 
celvcd fOii;htl.y lower v,tado.; In riL2 thni. did tliosf: In tlv? srnndurd 
preferoaco group and :; li;n If icaal ly lov/er grades Lhau did Iho^^n in 
the a.t. rnndom group. 

A) Finally, one rniiRt caution that It is somewhat risky to compare 
performance of the groups of students in the cngineerln)^, courses bt?- 
cause assignment to these courses was not random* 

The only clear pattern which emerged from an analysis of student 
grades in these six engineering courses is that students from the stan- 
dard random group had the lowest average grades in five of the six 
courses (that is, all but Engineering 1021 — Mechanics of Solids). 

In regard to physics courses, there was no apparent difference in 
the (small) numbers of engineering students from the four groups who 
enrolled in the Honors version of these courses. In regard to the t^ny- 
sics courses taken by most engineering students we obtained the follow- 
ing results: 

Random sections: In P213 (Electricity and Magnetism), stu- 
dents in the standard random group received lower average .scores than 
those in the a.t. random group. IJine students from the standard random 
group did not re*enroll in the subsequent semester. In P21A (Optics, 
Waves, and Particles) and P216 (Laboratory), there were no large differ- 
ences in average grades. 

Preference sections: In P213 there was no difference in aver- 
age grades; in P214 the standard preference students had substantially 
higher grades than the a.t* preference students. In P216 the standard 
preference students had somewhat higher grades than the a.t. preference 
students # 

Further, we analyzed the overall performance of the four groups 
of engineering students in their sophomore year. As the measure of overa 
achievement, we calculated the product of each student's grade point 
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nv;;l.-r. Cert, LP,;, valn.-s (»; 11u:t;.' proaucfw for fhc Fal l ^^r-nie.sLcr l':)7'5 
anr! Spria>; H-u-strf wo uLtain^'d the foUowLiu; re.suU.s: 

Uut lnn iho 1ir.-!t. f<j.:ic^sLer o£ th<.' si),.Uiin.orci yfai', ;^t( cK'al-..> froM 
tlK> a.t. ra-nlocu ser.tioiis liad .soi;H>what higher achiovc^ment than stan- 
d.ird rando;n student.;. Howevor after the attrition of niue of th^ 
standard random students and only one of the a.t. random students, 
there was little difference in the achievement of students from the 
two groups. On the other hand, during both semesters the standard 
preference students had higher achievement than the a.t. preference 
students. 

Stimmarizing the course achievement results, it is clear that the 
randomly assigned audio-tutorial students performed at least as well 
as (and generally better than) the randomly assigned standard students 
in their subsequent courses. However, among students assigned by pre- 
ference, the standard students performed somewhat better in some courses 
than did their audio-tutorial counterparts. That is to say, the rela- 
tions among the P112 grades of these groups of students were reproduced 
to some degree in their later courses. Audio-tutorial random students 
out-performed standard random students; standard preference students 
out-performed a.t. preference students. Wa believe therefore that the 
audio-tutorial students as a whole were not adversely affected when 
they returned to lecture-recitation-lab format courses. 

Finally, we will consider attrition rates of engineering students 
in each of these four groups. Please refer to Table 5 for the number 
of students in each of these four groups during each of the three 
semesters. 
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'i'abJ.o 





Students in 


(^roupt 


1 




ATll 


ATP 


STR 




Spring; '73 


57 


37 


96 


83 


KnlL '73 


51 


30 


86 


74 


Spring '74 


50 


27 


77 


72 


attrition 


7/57 


10/37 


19/96 


11/83 


% attrition 


12% 


27% 


20% 


13% 



Thus the rate of attrition among a.t. random students was lower than 
among standard random students; a higher rate of attrition occurred 
among a.t. preference students than among standard preference students. 
Therefore rates of attrition were higher for those groups which had 
lower achievement in PH2 (ATP, STR). Neither method was superior in 
regard to rates of attrition. 

Summarizing the longitudinal study: 

1) The audio-tutorial and standard students enrolled in the 
same selection of engineering courses; 

2) There were no apparent differences in achievement favoring 
rhe sLandard students in relation to the audio-tutorial students; 

3) Audio- tutorial and standard students had approximately equal 
rates of attrition. 



This Si'ctioii is cofupi..'i(;a of Lv;o i»n-t.s. The: fir.sL dciils with ob- 
servations of leariUng center instruction. Tho second deals with 
observtitions of rc^citations and of lectur«;?s. Observations were con- 
ducted to delineate the characteristics of the tuo methods of in- 
struction and thereby to make concrete the referent of this study 
and to increase its generalizability to other situations. 

A. Learning Center Observations 

The purpose of the observation of students and of tutors in the 
audio- tutorial learning center was to delineate Uhe proportion of time 
students and tutors spent in the many kinds of activities available in 
the learning center. To this end we developed a fairly simple obser- 
vation instrument and trained an observer in its use. (Please see 

Figure 3). 

The instrument was used as follows: 

During a period of 62 hours (distributed throughout the semester 
and representing each time. period and day of week approximately equally), 
the observer made nocations of the activities of the students and of the 
tutor. That is to say, every five minutes she noted how many students 
were engaged in each activity in Part 1 of the instrument, and which 
activity in Part 2 the tutor was engaged in. 

The results were analyzed as follows: 

1) In terms of the total number and percentage of students en- 
gaged in each activity under Part 1, both for the whole semester and 
for each of seven time periods during the semester; 

2) In terms of the percentage of time spent by the tutors in 

each activity under Part 2; 

3) In terms of the differences in time spent by the various 

tutors In each activity under Part 2. 

4) We also used the count of students present In the learning 
center during the observations to estimate the total number of hours 
students spent In the center. This number was then ccmpared with that 
obtained from tutor reports and from students* own accounts. 
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1' i {\uro 1 

(1) St ti^l'Mtt:. v;ork 

discussion ot course matc^rials 

2* performing or looking at a demonstrrition 

3. filling out a unit evaluaiion sheet 

4. viewing a film loop 

5. doing hoTiiework 

6. doing lab work 

7. doing miscellaneous, non-course work, etc. 

8. taking a pre-test 

9. doing a lab report 

10. waiting to talk to tutor 

11. reading text 

Study Guide Work 

12. reading objectives 

13 • working on portions other than problems 

14. just listening to tape 

15. looking at slide and listening to tape 

16. looking at suggestion slide 

17. working out problem solution 

18. looking at solution slide 

19. checking or correcting one's own solution 

20. copying solution without attempting problem first 

21. taking or chei:king a self-test 

22. referring to checklist 

(2) Tutor-Student work, or tutor alone 



1. 


tutor alone at desk or elsewhere, not in contact 




with students or equipment 


2. 


discussion of course material 


3. 


helping student with demonstration 


A. 


tutor fixing equipment 


5. 


discussion of homework 


6. 


helping with lab activity 


7. 


discussing miscellaneouii , non-course topics 


8. 


discussing a quiz 


9. 


helping with lab report 


10. 


walking around learning center, seeking questions 


11. 


out of room 


12. 


on phone 


13. 


listening to tape recording 


14. 


at blackboard 
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L. Stiu'eaL Al; L ivi c. u's . 

In nlJ, A/yA noi:if loa« vert- tPick' to uodcribe individual sluclont's 
nctlvU F.'ich tu.rnLion rofrrrcd t.o Che activity oE a fiinglc Htu- 

dout. Tu.' vaa.ior activities coii>priains 82% of the total were- at; foMown: 

(see Figure; A) 

1) 26% = lab work (//6) 

2) 20% = working out problem solutions in study p,uide (#17) 

3) 15% = listening to tape recordings (#14) 

4) 10% - doing miscellaneous, non-course work, entering or 

leaving the room, etc. (//7) 

5) 6% = discussion of coarse materials (gene. ally with tutor) (#1) 

6) 5% - looking at a slide giving a solution to a problem in 

the study guide (#18). 

Tlie observations of student activities were also analyzed for each of 
seven time periods during the semester. Tliese periods were as follows: 

1) Feb. 4-10 = first week 

2) Feb. 11-19 * period immediately prior to first prelim 

3) Feb. 20 - Mar. 8 

4) Mar. 9-24 

5) Mar. 25 - Apr. 2 - period immediately prior to second prelim 

6) Apr. 3-23 

7) Apr. 24 - May 16 ^ end of semester 

As Figure 5 indicates, there was a greater amount of student time spent on 
activities in the study guide relative to that spent on lab activities 
prior to each prelim and the final exam. The figures also indicate a de- 
crease in the average number of students present in the learning center as 
the semester progressed. These findings are in accord with informal ob- 
servations of the learning center. 
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BEST COnr AViUlABLE 



Time 

Pariod 



Avaraga 

of Students 
Observed 



Five-iMinut'e 
Observations 



to 

g 
o 

CD 

to 

X) 

o 



CD 
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CD 
Q. 




1 



Li 



1 



30r 




n 



1 



5.55 



5.82 



7.19 




10.36 



6.91 



1 1.19 



8.87 
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Student Activities 
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figure 5 
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2. Tutor Activities. 

KU.,v.-P per..oas f-orved an loaraunr. center tittorr.. Thes- ii.cludod 
seven instructors (two faculty lacnb.-rs aad five phy^Ucs i;rnau..te students) 
who taught audio-tutorial and standard rccitatloa sectiuaa, thrc^ graduate, 
siudents who taught standard lab and graded audio-tutorial studer.ts' lab 
reports, and a faculty member who had helped to develop the audio- 
tutorial materials (Dr. Ott.) 

One tutor was scheduled in the learning center at each hour of its 
operation. However, occasionally a second tutor would also be present. 
Thus there were 658 tutor activities noted during 619 observation periods. 

The major activities comprising 87% of the total were as follows: (See 

Figure 6):1) 33% - tutor alone at desk or elsewhere, not in contact with 

students or equipment (#1) 

2) 31% - tutor discussing course material [with a student] (#2) 

3) 14% = tutor walking around learning center, seeking 

questions (#10) 

4) 9% « tutor helping students with lab activity (#6) 

3. Differences Among Teachers. 

The various tutors behaved quite differently in the learning center. 
Four tutors spent about half the observed time completely out of contact 
with students (//I). Two spent about half the time discussing course ma- 
terial (//2). Two spent about 20% of the observed time helping students 
with lab (/^9) and three spent about 30% or more of the time walking around 
the learning center seeking questions 010). 
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BEST COPY AVAILABLE 
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Tutor Activities 

Figure ^ 
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vitlvial tiitoi:; in ench :<ctivity in V:\r-t 2 of thv. iastruneaL 

Tut or Act! V It MluLnuiu Maximum 

107. !>5Z 



1) tutor alone at desk or elsewhece, not in 

contact with students or equipment 

2) discussion of course material 

3) helping student with demonstration 
A) tutor fixing equipment 

5) discussion of homevork 

6) helping with lab activity 

7) discussing miscellaneous, non-course topics 
tt) discussing a quiz 

9) helping with lab report 

10) walking around learning center, seeking 

questions 

11) out of room 

12) on phone 

13) listening to tape recording 

14) at blackboard 



20 
0 
0 
0 
4 
0 
0 
0 
2 

0 
0 
0 
0 



51 
2 

22 
4 

24 

13 
0 
0 

36 

8 
4 



7 (only one 

tutor re- 
3 presented) 



4, Number of Student Hours in Learning Center. 

An estimate was obtained of the average number of students present 
In the learning center during the observation periods. That number is 
4794 3tudents/619 five-minute observation periods, or 7.74 students/ 
observation period. 

On the basis of counts made by the tutors of the number of students 
present at each hour in the learning center, the average number of students 
present was 6.78 — in good agreement with that obtained by the observer. 
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YIk^s,:; l.wo auiabc^vs r.oiitrasL with lutal auiabar of lioucs roporiu^l 

by rftuilv'iU"^; (»M ^'t iiiiL* cw<l::" wiiirh they vv\*rt* as^ t^nl to ciJiiplole each t imh 
lh»:y work<Hl ?n tlie U.-Miiln- contt r, The.;^* totaled 2725 hourf; for 1^0 
sladcuitu. AvLu'riged ovi^r the approxiiaately 6^f0 hours In which the It^araing 
coiiter was open, one obtaiaa only 4,3 Rtudevits/hour on the basisi of ''tline 
card*' reports. Thus students apparently failed to record about one third 
of the hours they spent in the learning center. 

From the student time cards > one calculates that the average stu- 
dent spent about twenty hours in the learning center during the semester 
or 1.4 hours/week. On the basis of tutor and observer counts of students » 
one can assume that this number is closer to thirty hours per semester or 
2.1 hours /week* 
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i>. 0.>fii.^rv,iL Ions t)l. i-i /.t urn au*l Ki?ciLtiLLou r4u;tr;ucCiou 

v:tiro iuttTi's; cvl in ub;;(.'rvlng both co^;ait i-vo r.nd bo»!uivlor;il :v^- 
poets of phyr»ics U^cruro «nul riHiMrition Instruction^ with our prinury 
concc*rn being the cognittvt! realm. However, a thorough review of the 
literature! did not reveal any observation instrument suitablci for ob- 
serving cognitive behavior in a college physics classroom. The author 
therefore developed such an instrument, patterning it, to ftome di^gree, 
on the Wright-Proctor System which was designed for use in the mathema* 
tics classroom. 

The instrument is intended for use in a physics classroom. It has 
three components — a list of ^'content" categories (please see Appendix 
B) which refer to the function of the statements being made by the 
teacher or by the students, a list of "behavior" categories (Appendix C) 
which refer to the non-verbal behavior of the teacher as well as verbal 
interactions between teacher and students, and a listing of the physics 
topics being discussed. (These topics were noted but not analyzed.) 
The two types of categories, content and behavior, correspond to an 
attempt to consider both cognitive and behavioral aspects of physics 
instruction. 

Furthernore, audio tape recordings were made of most of the classes 
which were observed. Tapescripts of a selection of these recordings 
will be analyzed as a further probe into the cognitive aspects of physics 
instruction. 

Use of the observation instrument requires some knowledge of the terms 
and forms of argument used in physics. Therefore a background in physics 
was a qualification for observers. Two persons with at least two years 
of college physics instruction were selected to be observers. They vere 
then trained to use the observation instrument by means of audio and 
video tapes of physics lectures and recitations. A brief listing of the 
instructions given to the observers is contained in Appendix D. The type 
of form used for recording the observations in included in Appendix E. 

2. Muriel J. Wright and Virginia Proctor, "Wright-Proctor System" in 
Mirro rs for B ehavior II; An Anthology of Observation Instrume nts (Phi la* 
delphia- Research for Better Schools, Inc., Spring 1970) p. 26-3. 

* Please see Appendix F for a report concerning the relation between 
cognitive aspects of instruction and student achievement. 
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:;orv4ir.-; v%as ii<'>; L^>ned. t;c> ohs-n vo rolcctOil loc turr* and rf»c Ltation classiL-r* 
III iM I2^ All obfierviHl lectures war^^ at the second of the two S"hc- 
duLiHl JiHtiite hours (l.o., 1?.:20 p.m.)* In a;UlUion, tht? obs^rvvts 
each attended fourteen dlfjcrent recitation s^-ctions at variouti cla:ss 
hours. Seven ti^achars taught these fourteen sections. Each teacher 
tnughr one standard and one audio-^tutorial recitation section. Standard 
recitation sections were attended by students in the standard method of 
instruction. These met for fifty minutes twice a week for fourteen 
weeks. Seven standard sections were observed twelve times during the 
semester, six times by each observer. The seven audio- tutorial sections 
were attended by students in the audio-tutorial method of instruction. 
These sections met for fifty minutes once per week. Each of these sec- 
rtons was observed six times, three times by each observer. Each section 
was observed for about 43% of the class meetings. 

The main functions of the observation instrument were to 1) specify 
the nature of lecture and recitation Instruction in this course, 2) de- 
termine the extent and kinds of variation among recitation instructors 
and between standard and audio- tutorial recitations, and 3) provide a 
basis for determining the interrelations among student achievement, stu- 
dent evaluations of teachers, and teacher behaviors. Each of these func- 
tions will now be discussed* 

1. Lecture Observations and Recitation Observations 

Lectures were observed by at least one observer and frequently by 
both observers. All twenty^six lectures were observed* In order to com- 
pare observers* categorizations of the lectures, observations of a set of 
thirteen lectures from one observer were compared with observations of 
thirteen other lectures from the second observer. The quantity analyzed 
was the total number of times each content category and each behavior 
category were noted for each lecture observed* For each content category 
and each behavior category, the two observers' totals for each of thir- 
teen lectures were compared using the methods of analysis of variance. 
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Lt. ;.:»v-<ilJ lu' liorcd, lK..,-vcr, t.li.iL codl .mi cMt..',;<u' io.i V, and 1'-) 

c»-)nibinea in th" an.i lysis dui' to the sltdlnrUy In the co-nir. svc 
U'vol or th-'.-5i.' c:iN',-;or I CM rtiirl (o t!ie fuvnll nunbiir of ohsorvatlons In 
each catt-Aory. (Caicjocy 8 refers to organising or fjtructuirini' rlu: i;;ri- 
terial, category 9 to npf^ciallz Lng to a particular c.use, and cat«p.ory 
10 to gonernlizing to a more InclusivG treatment from a particular case. 
Note that 9 and 10 just indicate how often these categories were used 
rather than the total amount of time spenc in these categories. Fcr ex- 
ample, the continuation of a generalization would be noted in other ca- 
tegories, depending on the further content of the discussion.^ 

There were no significant differences between the two observers on 
any of the content categories. There was, however, one behavior rnte- 
gory for which differences between observers were significant at .05. 
That category was category E which refers to erasing the blackboard, mov- 
ing a view-graph, or removing a transparency from the view-graph. ITius. 
cxrpntiag category E, the observers agreed in terms of lecture observa- 
tions. 

The mean values of the lecture observations are given in Table 6. 
Recall that two observations were made every minute. Thus a mean value 
of 11. 3 J for category F indicates that, on the average, 11.35/2 or 5.68 
minutes of each lecture were included in category F. (Category / rfftrs 
to the teacher performing a demonstration or showing a movie.) As indi- 
cated in Table 6, the predominant content category in lecture was cate- 
gory 3, the description of a physical system or process or demonstration 
or the use of a diagram or graph to Illustrate a process or kind of mo- 
tion. That is, lecture emphasized demonstrations. 

Second, in usage was category 7, deriving or explaining an equation or 
formula with explicit reference to its physical meaning or implications. 
The only other content category which received an average of more than 
six notations per lecture was the combined category formed from 8, 9, 
and 10. ITiese deal with organizing or structuring the material, general- 
izing, or specializing. 
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BEST COPY AVMIABIE 



Mc'»w> V;il.u.'S of l.ori iiri' Oh<u'rvat tons 



C o n t* 0 i > t C a r ci ^ jo r I o : 



0 


5.15 


P 


28.] 9 


1 


A . n/i 


A 


.1? 


2 


5.«1 


B 


26. HB 


:i 


35.04 


C 


.04 


A 


5.58 


D 


13.35 


5 


5.81 


E 


2.27* 


6 


1.96 


F 


11.35 


7 


18.4? 


G 


2.00 


8, 9, iO 


11.73 


H 


.38 


11 


1.65 


I 


5.58 






J 


5.04 



* observer disagreement significant at .05 

Table 7 

Mf»an Values of Recitation Observations 



0 
1 
2 
3 
4 
5 
6 
/ 

11 



Content Categories 
all sections a.t. standard 
n.98 



Behavior Categories 
all sections 



9, 10 



7.28 
4.82 

25.63 
8.44 
4.41 
7.44 

24.69 

.14 



6.51 



5.71 



9.15 



0 
A 
B 
C 
D 
E 
F 
G 
H 
I 
J 



37.50 
27.39 

12.79 

1.73 
.63 
17.33 
2.09 



* difference between methods significant at .05 
** difference between methods significant at .01 
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The i>rocIomLi\ant hchavua' caLe^'.ury was calvj^ory 0 which r.imply inai- 
catcr. that none of tho other behavior catejr.orius was appropriate. For 
lecture, caL'Ji;ory 0 pruiarlly inc:ludoj5 periods of t Ltuc in which tho :in- 
struct-or is simply speaking to the class rather than doing a demonst ra- 
tion, writing on the blackboard, etc. 

The second most comon behavior category, nearly equal in frequency 
to the first, is catcRory B, writing an equation or words on the view- 
graph (overhead projector), or showing a prepared transparency of an 
equation or of words on the vie\'-graph. 

Category D, drawing a diagram on the viewgraph or showing a prepared 
diagram on the viewgraph, was third in frequency. Category F, performing 
a demonstration or showing a movie, was fourth. 

Categories I and J both referred to the use of the Instant Response 
System. ITiis is a push-button system by means of which students' responses 
to questions are recorded. The system was available only in the lecture 
hall ured on Mondays. Therefore the observations listed under I and J 
represent behaviors observed at only half of the lecture classes. Thus 
although an average time of about five minutes per lecture is indicated 
in Ta^le 6 for categories I plus J, there was no time spent on these cate- 
gories on Mondays and about ten minutes spent in these categories in the 
Friday lectures. 

Recitation observations required a more complicated analysis than did 
lecture observations. There v re three primary factors in the analysis: 
1) method (audio-tutorial or standard), 2) observer, and 3) teacher. 
Tliat is, there were two methods, two observers, and seven teachers. The 
detailed results of the analyses of variance will be described in section 
2. However, the averages across teachers, methods, and ol servers will be 
cited here for comparison with the lecture observations. iTiese are listed 
in Table 7. These averages represent observations of one hundred twenty-six 
entire recitation sections. ITie averages were formed as follows: the average 
for eighty-four standard recitation observations and the average for forty- 
two audio-tutorial recitation observations were then averaged to form the 
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averiige for nil rof irnl iuns. Tims sLamlacd and aiul i o--tutor i al vocltu- 
tlons rocolvod equal weight in the final avornges. It should be noted 
that Individual Leacilicrs dif fer p i «nlf icnntly from tlio average on a 
number of thevse categories. 

As noted in Table 7, there were significant method difference;; for 
content categories 0 and the composite category 8, 9, and 10. Thus for 
these categories the averages for each method are listed rather than the 
overall average. The explanation for these differences is apparently 
that audio-tutorial recitations met only half as often as standard reci- 
tations but both types of recitations had an equal number of quizzes. 
Thus a higher proportion of class time in audio-tutorial sections was 
devoted to quizzes (which were categorized as 0). Apparently the time 
pressures which resulted from fewer class hours caused the teachers to 
use the higher order cognitive areas included in categories 8, 9, and 10 
less :. a.t. sections than in the standard sections. No averages are 
given for categories B, D, I, and J in Table 7 because view-graphs and 
the Instant Response System were not available to recitation instructors. 

As Indicated in Table 7, the predominant content category used in 
recitation was category 3. Category 7 was next in usage, nearly equaling 
category 3. Tliese are followed by category 0 in audio-tutorial recitations 
and by 8, 9, and 10 In standard recitations. 

In terras of behavior categories, the most common ones used in recita- 
tions were 0 (none of the others), A (writing an equation or words on the 
blackboard) , G (questioning or discussion with one student) , and C (draw- 
ing a diagram on the blackboard) . 

Lecture and recitations, therefore, tended to be similar entities in 
some respects. The same content categories predominated in lecture and in 
standard recitations, although lecture emphasized descriptions of systems 
(category 3) rather than derivations or explanations of formulas with re- 
ference to physical meanings or implications (category "') . The "average" 
recitation had approximately equal time devoted to these two areas. 

As far as behavior categories are concerned, category 0 predominates 
in both lecture and recitation. Category A is analogous to category B, 
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One notes t;:-.at Liu-.;.' l .it '-'.or i i-s ^.•>.: J a i.v;;>//- in I w > fu/,-.-^ 

iust ruK:t ion. T'LvAse r*'' V t:o writing', an cqu;gf iQn in words on tlu; black-- 
bonrd (A) or on ;i v ! c w-j-rai.li (B) . F-it-h form of i nrurucf i on incliulc-a 
about the fwimc amoiuil of drav,?in^ of diu};if^>i3 on the blackboard (C) or on 
the vicw-^jraph (D) . However, lecture included a far greater cmphasit; on 
demonstrations (F) th.^n did recitations. Conversely, recitations tended 
to emphasize qiie.stioni:ni or discussion with individual students (G) and 
this category was infrequently noted in'*iecture. Note that this is to 
be expected considering the purpose of the two forms of instruction. 

Thus lecture and recitations on the average had many features in 
comiron. However, as will be detailed in the next section, the average 
values given for recitations in Table 7 do not represent tb3 diversity of 
teaching styles which were observed in the recitations. 

2. Observation of Recitation Instruction: Method, Observer, and Teaclier 
Effects 

As suggested earlier, the analysis of recitation observations was 
far more complex than the analysis of lecture observations. The research 
design for recitation observations had three factors ~ method, observer, 
and teacher. There were two methods, two observers, and seven teachers. 
Each teacher was observed eighteen times, six times by each observer in 
the teacher's standard recitation section, and three times by each observer 
in the teacher's audio-tutorial recitation section. Thus there was a total 
of 126 observation periods, equally divided among teachers and among 
observers. However, audio-tutorial classes were represented in forty- two 
(1/3) of the observations, and standard classes were represented in eighty- 
four (2/3) of the observations. 

In analyzing each separate content and behavior category we employed 
analysis of variance techniques. The quantity analyzed in each case was 
the total for a catc^^ory for an entire recitation period. 

A sample analysis of variance table, indicating the sources of 
variation and degrees of freedom, is presented here to indicate the 
factors and the two- and three-factor interactions which were analyzed. 
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BEST COPY JliMUBLE 



Sourco 



Observer 1 

Tccichor 6 

Method by Observer 1 

Mathod by Teacher 6 

Observer by Teacher 6 
Method by Observer by Teacher 6 

Error 98 



Total 



125 



For categories in which method by teacher interactions were found 
to be significant, we revised the analysis of variance so as to con- 
sider differences between recitation sections. That is, teacher and 
method factors were grouped into a factor called "section". The analysis 
of variance table for this alternate analysis is indicated below: 



A significant method by teacher interaction and thus a significant 
section effect means that individual teachers differed in the ways they 
taught their audio-tutorial sections relative to the ways they taught 
their standard sections. 

The remainder of this section will treat the content categories and 
then the behavior categories, looking at method differences, observer 
differences, and teacher differences. Interactions will be mentioned if 
significant. Ttie level of significance was set at .05. 



Source 



df 



Observer 
Section 

Observer by Section 
Error 



1 
13 
13 
98 



Total 



125 
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a) CaiiLeut Catfi^or lus 

CaLcj'.ury .41-i:-; 4u>t . 1 y/ed because ovaa^ -'-i Vi' -re y^^^' ^" 

la I" e iVJ.t'5Ai!i\^ 

For cato{T,ory 1, Hierc; was a method by teacher interaction siguificaat 
at .01. Thus this category was analyzed by section rather than by method 
and by teacher. Tl>eso results are discussed under section diffcroaces. 

No other interactions were yignif leant. 

Method Differences 

The only method differences were as follows: 

For category 0 there was a difference between methods, significant at 
.05. Category 0 was noted more often for ludio-tutorial than for standard 
sections. As mentioned in the section comparing lecture and recitation, 
this difference is attributable to the fact that audio-tutorial .ections 
spent proportionally more time taking quizzes than did standard sections. 

For the combined category 8, 9, 10 there was a difference between 
methods significant at .01. Category 8. 9, 10 was more common in standard 
than in audio-tutorial recitations. Apparently teachers reacted to the 
fact that audio-tutorial sections had half as much recitation time by us- 
ing these higher order cognitive categories less often in audio-tutorial 
sections than in their standard sections. 

Teachers appear to have treated audio-tutorial and standard sections 
in fairly similar ways. However, they deemphasized higher order cognitive 
categories 8, 9, and 10 in audio-tutorial sections. 

Observer Differen ces 

Observer differences were noted for category 1 (significant at .05)^ 
for category 3 (significant at .05)jand for category A (significant at 
.01). Tliese differences indicate a need to clarify distinctions among 
categories. Fortunately, however, these observer differences did not 
significantly interact with method or teacher effects. 

Teacher Differences (Please refer to Table 8.) 

For purposes of comparison teachers will be referred to as I, II, III, 
IV, V, VI, and VII. Significant differences were found In two content 
categories. 



- 43 - 



BEST COPY AVAILABLE 

Tcvichor diffoiMncGS y i p.n I f ic.ant. at .01 were found for cat egory A 
which n-lsTto;.! to r.l..'Uinr, or rophrnsin;^ a problem. Tt^achers T and IV 
had icautly fowcr observations in this catf^ory than teachers TTl, 

V, and Vil. (The Ncy-Tian Kcnls analysis was used to test dlf f eronces. ) 
Teacher differences significant at .01 were found for category 7 which 
is concerned with describing or cxplainins an equation or formula with 
explicit reference to physical meaning or implications. Teacher I was 
significantly higher on this category than teachers II, III, V» or VII. 

Tliese results for categories 4 and 7 indicate that some teachers 
(I and perhaps IV) emphasized the statement of problems more than others 
did (III, V, VII, and perhaps II). Further they indicate that those 
teachers who emphasized problem statements, de-emphasized problem solu- 
tions relative to the other teachers. Finally, teacher VI did not empha- 
size either category relative to other teachers. 

One should note that all teachers had more observations in category 
7 than in category 4. However, the ratio between the observations in 

these categories varied from 

32.44/ 2.94 11.0 for teacher I to 
22.67/10.67^ 2.1 for teacher V. 
Table 8 

Mean Values of Content Categories for Teachers 
Teacher I II HI IV V VI VII 

Content 
Categories 

9 

1** fitamlard 
1** a.t. 

2 
3 

4* 
5 
6 

7* 

8,9,10 

* Teacher difference significant at .01 
** section difference significant at .01 



3.83 


9.56 


6.78 


7.83 . 


10.83 


10.67 


8.83 


4.25 


6.15 


8.17 


4.75 


4.75 


7.40 


6.25 


12.83 


13.50 


4.33 


4.83 


6.83 


12.33 


5.50 


4.62 


4.28 


3.33 


4.33 


5.72 


6.11 


4.50 


26.28 


25.94 


29.61 


27.44 


26.28 


19.61 


28.06 


2.94 


9.61 


10.33 


4.89 


10.67 


8.56 


11.94 


4.22 


9.00 


4.44 


4.94 


3.89 


3.50 


3.67 


4.28 


5.94 


6.33 


9.50 


6.61 


11.28 


7.50 


32.44 


19.28 


20.22 


28.17 


22.67 


26.94 


23.55 


9.78 


10.78 


7.50 


6.56 


7.39 


6.94 


7.11 
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Catc<\ory 1, whirli liwolvo:^ KtatenuMttt; conc:ernlna course and c lass- 
room admiaistracion, v;;iw nnulyztid by recttaLioii Rcctioa duo to a sig- 
nificant method by rc-achor interaction. Differences between sections 
were significant at .01. All three of the sections which were high on 
category 1 were audio-tutorial sections. These were taught by teachers 
I, II, and VI. 

Sections significantly louer than these on category 1 were the 
standard sections taught by I, IV, and V, and the audio-tutorial sec- 
tions taught by III, IV, and VII. In addition, the standard section 
taught by II was significantly lower on category 1 than teacher II 's 
audio-tutorial section. 

Apparently a number of teachers used category 1 proportionally more 
of the time in their audio- tutorial than in their standard classes. 
However, other teachers apparently de-emphasized this area in their 
audio- tutorial classes. 

b) Behavior Categories 

Categories B, D, I, and J did not pertain to recitation instruction. 

Inter a ctions 

There was a method by teacher Interaction for category A which was 
significant at .10. For this reason category A was analyzed by sections. 

There was an observer by teacher Interaction for category F which 
was significant at .01, and an observer by teacher interaction for cate- 
gory H which was significant at .05. Tlie observer and teacher main ef- 
fects for these categories will therefore be considered in the light 
of these Interactions. 

Method Differences 

There were no behavior categories for which method differences were 
statistically significant (at .05). 

Observer Differences 

For both category F tnd for category H there were observer dif- 
ferences significant at .01. (F is concerned with demonstration, H with 
discussions between the teacher and two or more students.) Observer 2 
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luicd r".t.c'i'.ory F nior.' ofrcni th.in ri ul obyorver 1. Ohfiervcr 1 used cate- 
gory II Ti'orv often r.h;in did Qhr,crvc.r 2. However, these differences woro 
not consIiiCtniL across teacbcrii as indicated by the presence of obscrvtjr 
by teacher interactions. 

Teachc r Pi f f e r ences (I'lease refer to Tabic 9.) 

Teachers differed on category 0 (none of the other behaviors), nig- 
nlficanU at .05. The teachers highest and lowest In this category were 
teacher VI and teacher VII, respectively. 

Category A (writing an equation or words on blackboard) is described 
under Section Differences due to the method by teacher interaction. 

Teachers differed on category C (drawing a diagram on blackboard), 
significant at .05. Teacher VI was significantly lower on this cate- 
gory than was teacher E . 

Teachers differed on category G (questioning or discussion with one 
student) significant at .01. Teacher I used this category significantly 
less than all other teachers. Furthermore, teachers IV, II, and VI used 
this category less than did teachers V and VII. 

Category F and category H each reflected significant differences be- 
tween teachers but were complicated by observer by teacher interactions. 
However, for category F observers agreed that teacher VII was signifi- 
cantly higher than all the others. There was disagreement on the rela- 
tive amount that the other teachers used category F, but all were sub- 
stantially lower than teacher VII. 

For category H there is a substantial disagreement among observers 
about individual teachers which makes teacher differences unreliable. 
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Mv>an V.iliio.^ of HcHavJor Catt'i'.orics for 'i't'acliori-j 



Trnrh. r I if I IT TV V VT VII 

iic!hrivif»r 

0 39.17 A2.33 31.28 38.22 32.33 A4.56 31.11 

A^'** standard 35.25 26.25 24.67 35.08 22.25 29.33 28.67 

A*** a.t. 28.50 19.50 30.00 24.00 24.83 31.00 24.00 

C* 15.00 16.22 11.33 12.78 14.94 9.67 11.33 

E 1.61 1.50 2.50 1.72 1.56 2.17 1.44 

F** .33 .78 .22 .06 .22 .28 2.83 

G** 5.72 14.50 20.56 13.56 23.67 14.72 25.28 

H (.72) (1.83)(3.39) (.89) (3.61) (1.39) (2.33) 



* teacher difference, significant at .05 
** teacher difference significant at .01 
*** section difference significant at .01 

Summarizing significant teacher differences: 

1) for category 0, VI used this category more than VII did; 

2) for category C, VI used this category less than II; 

3) for category F, VII used this category more than all other 
teachers 

4) for category C, I used this category less than all other 
teachers and In addition, IV, II, and VI used the category 
less than did teachers II and VIT. 

Section Differences 

Category A was analyzed by section due to the method by teacher 
interaction. The difference between sections was significant at .01. 
The difference found was that teacher II used this category significantly 
less for his audio-tutorial section than teachers IV and I did for their 
standard sections. Other sections did not differ significantly. There 
was no major pattern of differential usage In audio-tutorial vs. stan- 
dard sections. 



To complete this section on recitation observations, we wish to 
tress the differences in teacher behaviors which make it difficult to 
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dicaro lUv raaryfi of ncvnn vjlitoi; on oach of On- coniont and bohavrior 
c;>t *'>;or I('s for thr { u^nAii-rt^. •Tht^.c** r.ve };ivi»n in Table K) in mlniitcT. 
por category rather t lian in ol)St'rval ioas pec ratcigory. 

TahJe 10 

Rango of K.^.aa Valvte?; for Toacbcr Observations 
(Valaes in Number of Minutes) 



Content Categories Behavior Categories 





Minimum 


Maximum 




Minimum 


Maximum 


0 • 


1.92 


5.42 


0 


15.56 


22.28 


1* 


2.13 


6.75 


A* 


9.75 


17.63 


2 


1.67 


3.06 


C 


4.84 


8.11 


3 


9.81 


15.81 


E 


.72 


1.25 


4 


1.A7 


5.97 


F 


.03 


1.42 


5 


1.75 


4.50 


G 


2.86 


12.64 


6 


2.14 


5.64 








7 


9.64 


16.22 








8,9.10 


3.28 


5.39 









* given Tor sections rather than for teachers 
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Clearly there wa3 a broad variation in teaching styles used in these 
recitations. The morf: striking example is category G v/hich monitors how 
much teacher-student interaction occurred. Clearly teacher patterns 
varied from one in which very little teacher-student Interaction occurred 
(a lecture style), to one in which about 25% of a typical class period 
was devoted to teacher-student interactions. 

3. Relations of Recitation Observations to Student Achievement. 

3 

As has been emphasized by Rosenshine and Furst, it is important to 
try to relate the results of classroom observations to student outcomes 
such as course achievement. This step is important so that the field of 
education can eventually accrue evidence as to the value of various teach- 
ing characteristics, both cognitive and affective. At present only a small 
number of affective characteristics and even fewer cognitive characteristics 
have consistently been found to be related to student achievement. 

3. B. Rosenshine and N. Kurst, "The Use of: Direct Observation to Study 
Teaching", ^r. <it*rnjit\ HnnHhonk pf Roflea rch on Teaching , ed. by R.M.W. 
Travers, (Chicago: Rand McNally College Publishing Co., 1973), p. 122. 



dont acliiw.MCMU, oik- iKist kooi' in initi.l the. I'mnn of Mr? proscpt re- 
search. 1) Kirwr, a snail p.roup of fcachorf: w:iP involved .in tbo atudy 
siwon tcr.f.luTf. ill i cu' itAoii rcrilatlon rw-ct ioiin . 2) Ucc. r tal ion was o.ily 
one iiuu factional ccnipouL-nU of t.lu« cnur.-^c. StudcnL-, with vmsaL i sf -ictory 
rocltatiou instruc.lors could attompC to learn tho course material through 
lecture, the learning contor, texts, and go forth. 3) All results are 
correlational rather than causal. That is, this study did not attempt 
to alter teacher behavior and then to determine diCferenceiS in student 
achievement. 4) Our measure of student achievement was fairly narrow. 
Due to possible differences in grading practices for the recitation por- 
tion of the grade, and to the fact that lab work was fairly independent 
of recitation work, we used exam grades as the measure of achievement in 
this part of the analysis. 

Within these narrow bounds we analyzed the correlation between teacher 
behaviors and student achievement. 

As the measure of teacher behavior V7e used the total number of obser- 
vations in each observation category for each recitation section. Tlie 
totals arc given in Tabic 11 for audio-tutorial sections and in Table 12 
for standard sections. In Table 11 all totals have been multiplied by a 
factor of two since thor.e sections wc-re observed half as often as were the 

standard sections. 

As the measure of student- achievement , we used exam grades, adjusted 
for SAT math scores and math pretest scores. For each student we formed 
a total exam grade from ttie average of the two prelim grades plus the stu- 
dent's final exam grade. The average value , of this total exam grade for 
each recitation section is giver in the first column of Table 13. Using 
analysis of covariance, we obtained adjusted values of the total exam 
grade for each recitation section. The covariates were the SAT math scores 
and the math pretest scores. The adjusted total exam grade, the average 
SAT math score, and the average math pretest score are reported for each 
section In Table 13. 
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Observation Category Totals 
for Atulio^'futorlal Roritatlon Sections* 



Teacher: 


I 


II 


111 


IV 


V 


VI 


Vll 


■ V 

Laccgory 
















0 


56 


246 


110 


116 


118 


212 


148 


1 


154 


162 


52 


58 


82 


148 


66 


2 


100 


58 


40 


38 


70 


46 


82 


3 


254 


220 


340 


362 


350 


170 


318 


A 


44 


104 


148 


74 


126 


100 


118 


5 


56 


96 


46 


24 


40 


42 


26 


6 


56 


52 


100 


104 


72 


164 


100 


7 


354 


184 


286 


294 


306 


348 


284 


8.9,10 


66 


98 


80 


56 


78 


66 


36 



0 


524 


598 


396 


426 


376 


604 


352 


A 


342 


234 


360 


288 


298 


372 


288 


C 


162 


166 


122 


170 


174 


n 


124 


E 


16 


18 


28 


16 


16 


20 


16 


F 


6 


0 


6 


0 


2 


2 


26 


G 


72 


182 


252 


220 


316 


188 


346 



* All totals have baen multiplied by two in order to make these totals 
comparable to those for standard sections which were observed twice as 
often as audlo-tutorl ^i. sections* 



Table 12 







Observation Category Totals 










for 


Standard 


Recitation 


Sections 








Teacher 


I 


II 


III 


IV 


V 


VI 


VII 


Category 
















0 


41 


49 


67 


83 


136 


86 


85 


1 


51 


74 


98 


57 


57 


89 


75 


2 


33 


48 


40 


59 


68 


87 


40 


3 


346 


357 


363 


313 


298 


268 


346 


4 


31 


121 


112 


51 


129 


104 


156 


5 


48 


84 


57 


77 


50 


42 


53 


6 


49 


81 


64 


119 


83 


121 


85 


7 


407 


255 


221 


360 


255 


311 


282 


8»9.10 


143 


145 


95 


90 


94 


92 


110 



0 


443 


463 


365 


475 


394 


500 


384 


A 


423 


315 


296 


421 


267 


352 


344 


C 


189 


209 


143 


145 


182 


128 


142 


E 


21 


18 


31 


23 


20 


29 


18 


F 


3 


14 


1 


1 


3 


4 


38 


G 


67 


170 


244 


134 


268 


171 


282 
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Duo. lo tHo fad tint I ho s;i!i'.-' I l.-.ji'1.o t;ni:-.;a o,-.' .itrl i o- tu i tu" i al 
roi it;it.i<Mi ;uul oiu> .stanJara n>ciLatL()n .'ujetiion, vo annlyy.oa rho (U;t^ 
j<.>i>aral.' V ic.i: lli. .m ! ! attM' i /il .v.rc r iot,:, aiul r'>)- rh,- stMidara .-..-cf Ion ■> 
nnd thoii couiMartid th..: tei-ults of these two analysoH. Thus for «ach set 
of sov<fit rcritritlon soction.-^ and for each obst'cvation catoftory wo. oh- 
tained tlio correlation coefficient assesslnji the dt-gree of relationship 
between t«)tar numbers ot observations and adjusted exam totals. Tlicr.c 
coefficients are listed in columns one and two of Table 14. Finally, we 
averaged the two correlation coefficients for each observation category 
using Fischer's r to z transformation. These average correlation coef- 
ficients are listed in the third column of Table 14. 

The correlation coefficients listed in Table 14 indicate trends in 
the data. Of course due to the small number of teachers involved these 
trends are only suggestive. Six correlation coefficients had the same 
sign in both groups of recitation sections and were moderately lar^^e. 
Two categories, 1 and 0, were negatively correlated with adjusted exam 
scores. Four categories. 3, 4, C, and G, were positively correlated with 
exam scores. These data suggest a negative relationship between an em- 
phasis on course and classroom administration (1) and student achievement 
on exams. Low usage of behaviors listed In the behavior categories was 
also nei;acively correlated with student achievement. This type of teach- 
ing would be categorized in behavior category 0 and would be character- 
ized by a large amount of lecturing without props, writing on the black- 
board, or interaction with students. On the other hand, describing a 
physical system or process or a demonstration (3) , stating a problem or 
rephrasing a problem (4) , drawing a diagram on the blackboard (C) , and 
questioning or discussion with one student (C) were each positively cor- 
related with student achievement. Note that categories 3 and C often de- 
scribed the cognitive and behavior characteristics of a given teacher 
behavior, as were the categories 4 and G. Thus the four positive cor- 
relations found here are probably reducible to two — between the use of 
behavior which would be categorized as 3C and the use of behavior that 
would be categorized as 4G. Similar remarks apply to categories 1 and 
0, since these often were used together to categorize a given teacher 
behavior. 
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Mil I -V'-r; 












for t'.irh U ' M t : 


ill t 1 S-''.' 1 i c;' 












Adjusted 




Mnth 




Numbc^r of 


Total Kxam 


Total Kxam 


Math 






St uJ:Mil!' 


Scot o''^'* 


Sc i)r. -••'>* - 


SAT 


Errors; 


I 


AO 


1,/. w • ' /. 




690 


2.42 


11 


1 A 




97.91 


6S7 


3.07 


III 


21 


131.95 


126.42 


694 


1.81 


IV 


18 


107.39 


114.83 


661 


3.67 


V 


19 


125.08 


1 9 7 7 A 


A7Q 
O / ? 


Til 


VI 






110.23 


691 


2.41 


Vll 


1 ft 


111 9fi 


110*52 


690 


2.67 


Teacher 












I 


1 7 
1/ 


lAQ Of. 


112.59 


689 


3.71 


T T 
II 




190 OA 


129.37 

X^ y • J w 


672 


2.48 


T T T 

L L 1 


27 


116.22 


118.31 


669 


2.78 


IV 


23 


119.65 


118.18 


698 


2.70 


V 


19 


127.05 


126.67 . 


683 


2.58 


VI 


18 


113.53 


111.58 


698 


2.61 


VII 


19 


117.61 


115.30 


695 


2.47 



* The same criteria were used for including students' data as were 
seated on page 20 of this report. 

** Scores differed significantly across the fourteen sections (level 
of significance - .05.) 

, Table 14 

Correlation Coefficients for 
Adjusted Total Exam Scores vs Observation Categories 



Category 


Audio-tutorial <n=7) 


Standard (n=7) 


Average 


0 


-.788 


.236 


-.397 


1 


-.545 


-.143 


-.354 


2 


.055 


-.011 


.025 


3 


.612 


.224 


.438 


4 


.224 


.376 


.300 


5 


-.514 


.617 


.081 


6 


-.010 


-.098 


-.055 


7 


.571 


-.564 


-.064 


8,9,10 


-.085 


.191 


.000 


0 


-.619 


-.170 


-.422 


A 


.548 


-.611 


-.046 


C 


.086 


.663 


.414 


E 


.284 


-.441 


-.092 


F 


-.031 


-.025 


-.025 


G 


.202 


.311 


.254 



* Averages were formed using Fischer's r to z transformation 



A.i u.Lt;L:«io;>cu (.aclior, aiialy.sji; u-ii\l a faii'ly naci.\>w i<.:li;a- 

tioa ol f;t.:uJinil ach i c^vcnent , i*v.^ c^xain j^rac1t*Li» nad hnvy isnorcd otlu^r 
pc)r;ail>"!" liK^asitret; ol" i)hy:uc\-^ adi i o\/cTieiU, . in particular iL ha:: l!;^n(K.pd 
the clevulopinont of Inj^Jior ordcir cojiiiilive abilities rucIi as the ability 
lo gcinorallzc or to spocinllze, l)i?-VGlopmetit of thes^i higher order cor,- 
nitlve abilities might be associated with emphasis on concepts or theories 
(category 5) or with generalizing or specializing (category 8, 9, 10). 

4. Relation of Recitation Observations to Teacher Ratings 

Finally, we attempted to relate student ratings of instructors to 
the instructors' use of the various content and behavior categories. We 
sought to detertniae whether any general pattern would emerge relating 
teaching behaviors or teaching styles to student ratings of teachers 
The ratings used were based on those developed and reported by Hildebrand, 
Wilson, and Dlenst.^ Students were asked to (anonymously) rate their 
recitation instructors on each of the following rating scales • 
The teacher: 

Rl, Has command of the subje^it, contrasts various points of 
view, discusses current developments, and relates topics 
to other areas of knowledge* 

R2. Makes himself clear, states objectives, summarizes major 
points, presents maLerlal in an organized manner, and pro- 
vides emphasis, 

R3. Is sensitive to the response of the class, encourages 

student participation, and welcomes questions and discussion* 

R4. Is available to and friendly toward students, is inter- 
ested in students as individuals, and is valued f'or advice 
not directly related to the course. 

R5. Enjoys teaching. Is enthusiastic about his subject, and 
makes the course exciting. 

Ratings could run from 1 to 7, with 1 indi-^ating that the items was "not 

at all descriptive of the tcacherV and 7 indicating that the item was 

"very descriptive/* 

* Please note that our obycrvatiou program focused on certain character- 
istlrs of clat;sroom toacliing and necessari ly Ignored other relevant data 
which may affect studc^nt ratings of teachers, such as the number of times 
the teacher was absent from class* 

4. M. Hlldebrand, R.C* Wilson a^d E.R, Dlenst, "Evaluating University 
Teaching" (Berkeley Center for Research and Development In Higher Educa- 
tion, University of California, 1971), p* 23. 
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Ucwults were aivily/vd by soctioa rather than by teacher. Signifi- 
cant diffurenccs (at .01) were found for each raring scale. Mean values 
of section.s* rathiju'. are ^Iven in Tabic l^. 

Scctloob w«re usually rated In about tlie same; order on the five 
scales. The pattern was as follows: 

From his a.t. section, teacher VI received tho lowest or second 
lowest ratings of all teachers on all scales. 

Tlie, a.t. section of teacher I and the standard section of teacher 
Vtl rated their teachers very highly. These two sections generally gave 
very high ratings on each scale. 

As was the case with the analysis of observation-achievement rela- 
tionships, our analysis of observation-ratings relationships was limited 
by the smsll number of sections studied. Nonetheless, a type of pattern 
did emerge in relation to teacher ratings. 

Table 15 

Mean Ratings of Recitation Instructors 
Maximum Number of 

Teacher 



5 L 

TO II 

ti c III 

2ll IV 

o <u V 

•o VI 

< VII 



I 

II 

TO I" 
V" (2 IV 
c1 O 

t> -H V 

5 o VI 

<0 TO VII 



ratings received* 


Rl 


R2 


R3 


R4 


R5 


22 


6.27 


6.73 


6.36 


6.18 


5.96 


13 


3.08 


6.08 


6.00 


5.50 


5.69 


19 


4.50 


5.47 


5.79 


5.07 


5.00 


13 


5.85 


6.23 


6.46 


5.67 


5.58 


10 


5.50 


5.20 


6.30 


5.70 


4.80 


19 


4.63 


4.95 


4.58 


4.12 


4.17 


20 


5.65 


5.45 


6.15 


5.47 


5.20 


12 


5.75 


6.25 


6.25 


5.92 


5.83 


28 


5.61 


5.22 


5.07 


3.92 


4.52 


24 


5.09 


5.54 


5.75 


5.09 


5.13 


19 


5.00 


5.95 


5.47 


5.00 


4.74 


12 


5.08 


5.75 


6.33 


5.17 


5.42 


14 


4.93 


5.50 


6.14 


4.85 


4.93 


16 


6.44 


6.19 


6.31 


6.00 


5.81 



* Total numbers of responses differed for each rating. For Rl, N=«238; 
R2, N=239; R3. N-241; R4. N=223; R5, N-237. 

The two teachers receiving the highest ratings, teacher I and 
teacher Vll^were quite different from each other in style. Teacher I 

featur-id a lecture style emphasizing problem solutions (7) and teacher 
VII featured a questioning style (G) emphasizing problem statement (4). 
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ih\ tiu:' olhtM" iiatui, tlK* ttiachCi who rectuvtul lltii lowest rauin>;s, 
teacher Vf, v^/^u: in the ulddlo on caLegory A, caLoijory 7, and c^itCKory C. 
He hij;h on r.atA?;;;ory 6 whu h i nclicai.os an en:pha;ru; oa mixthcv^t i.c.i\) 

explanations of piob-l»*m-^ and 0 which indicates that he simply talked 
without Lining any writtt^n support* Finally, he was lowest among teachers 
on cate.porlos 3 and C which refer to the description of physical pheno-- 
mena and the use of drawings on the blackboard. 

These results, although quite sketchy, appear to indicate that 
the teachers who used highly developed teaching styles were rated signi- 
ficantly higher than was the teacher who used a fairly undeveloped style 
of teaching. 



ERIC 



This ri'SC'.ii-i'h bt-ou toiiducLcHl in part to assist in tho dcvt'LoprooiiL 
of guixle lines for course decisions. These decisions are In the following 
arens : 

1. methods of instruction; 

2. procedures for assigning students to methods; 

3. coordination of methods of instruction; 
A. the operation of the learning center; 

5. selection of teaching styles for recitation. 

As far as selection of a method of instruction is concerned, both 
methods lead to similar levels of achievement for the majority of randomly 
assigned students. However, students with relatively low math achieve- 
ment and math aptitude may profit more from the audio-tutorial method than 
the stand method. Conversely, students with very high math achievement 
and mat*^ . t:ude may profit more from the standard method than from the 
audio-t jrial method. 

If both methods of instruction are to be used in a course such as P112, 
one should keep these factors In mind in the selection of students. If 
only one method is to be selected for all the students in a course, the 
students* math achievement and math ability characteristics ought to be 
major criteria in the selection of a method of instruction. 

In regard to selection of a procedure for assignment of students to 
methods, assignment by preference has many drawbacks In relation to random 
assignment. It is more difficult to assign by preference. When pre- 
ference is used as an assignment procedure, some students are assigned 
contrary to their preference so as to balance class sizes. This leads 
to confusion and dissatisfaction. When preference Is the assignment pro- 
cedure, students who select the audio-tutorial method appear to be over- 
confident in regard to their success in the course. They under-utillze 
the available materials and achieve significantly less than do students 
who choose the standard method. 
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att UucIgs to tlio int-t!ioJ oT Inst, l uctiioo Lht^y receive* llian i:^ tudtuil s v;!iO arc 
r.'uidorily «! i j^^hlhI • K 'V^MTb.:' hvss , slucI.Mit v;!io nre ranUoml.y asfiiv.nod 
genOTalty rvinnln in iho ];;v»thod to wliicli thoy are asyiiynud. 

On balance > random a.ss I^iamont appears Lo be preferable to asMignmont: 
by preference, particularly in the light of the consistently lower levelr> 
of achievcMiit^nr obtained by the a,t. preference group, the failure of a,t, 
preference students to utilize the method they rate highly, and their 
failure to compensate j.or lower performance in P112 by performing better 
in their other courses. 

If both methods of instruction will be used In a course, tha best 
assignment procedure would seem to be as follows: 

1) assign students at the upper end of the math ability and 
miith achievement range to standard instruction; 

2) assign students at the lower end of the math ability and 
math achievement range to audio-tutorial instruction; 

3) randomly assign students in the middle range. 

In regard to the coordination between the two methods of instruction, 
attention is needed in order to avoid differing emphases and coverage in 
the two methods. Such differences result in student and teacher dif;snti3- 
fiiction. 

In regard to the learning center operations in P112, some difficulties 
are apparent. Observations indicate that periodically there were very heavy 
loads on the lab facilities of the learning center, causing breakdown of 
apparatus and overflow into other facilities. These problems should be 
alleviated by increased technical assistance in the lab. Furthermore, 
certain instructors working as tutors Interact little if at all with 
studtints in the learning center. Due to the fact that there Is very 
little student-to-student interaction in the learning center, some effort 
is needed on the part of tutors to Interact with students and thereby to 
personalise teaching in the learning center* 
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In rttgard to rccit aLion instrurt ion, obsoivacions support llic 
followinjJi: 

Less reci-taivlon tlmo slioultl be clovotud to quiazos in th.- aadio- 
tutorial so.ctlons than in the standard sections. This wouid remove 
some of the time prt-ssure felt by the audio-tutorial teachers Bnd might 
result in increased discussion at higher order cognitive levels. 

If recitation instructors arc interested in improving student 
attitudes toward their teaching, they might be advised to develop a de- 
finite teaching style. They might, for instance, emphasize either ex- 
position of material or questioning and discussion. The teaching &tylG 
which is developed should include adequate use of written supports such 
as equations and diagrams. 
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!-)'0(jijL'ntly , cot.iparisoiVj of studont achicvGrnenL in two or more 

•in-Structi(;na1 troa Uian ciro roportod lo result in no signiricant difFer- 

1 ? 

cnces between trcatrnsnts . llov/evsr, as has been cniphasized recently, 

In such studios one also ought to loo!; for interactions botv/een student 
traits (including aptitudes) and treatments (e.g., instructional irathods). 
If such interactions are found, one treatment ir,ay be better for some 
students having certain levels on the trait in question, and the other 
treabnent nay be better for students, having other levels on that trait. 
It is clearly important to identify such interactions so as to guide fa- 
culty and students in selecting instructional mathods. 

Vie have used a trai t- treatment interaction approach to analyze 
the achievement of first year college physics students in tv/o instruc- 
tional treatments — an- audio-tutorial treatment, and a lecture-recitation- 
lc»boratory treatment. We have found indications of a disordinal* inter- 
action betv/een the treatments and mathematical aptitude (as naasured by 
the mathematics portion of the Scholastic Aptitude Test) and between the 
treatments and mathematical achievement (as measured by a pretest of our 
dDsign). The dspendent variable v/ as the students' achievement in the 
course as measured by course final grade. 

'^■'A disordinal interaction is one in which regression lines for the two 
trcdfcments cross within the observed range of the measured trait. 
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BEST Wl wmXBUt. 

IcvL'l physics course ofUiro.d to ttpproxiriatoly 575 cngtneorincj and pl^ysics 
rujors at Cornell Unlvorsity. About. 98;'. of ths students v;oro freshrren, 
91% v/era enrolled in enginooring, and 95% v/era nales. Ibst studt^nLr. v/ora 
taking the course becau3e it was required. The subject matter of the 
course includad classical nischanics, special relativity, and heat. Ona 
seiroster of calculus was a prerequisite for registration in tha course. 
The course textbooks were Fun d aingnt als of Physics by David Halliday and 
Robert Rosnick and Space and Tirna in Spe cial Rolativtty by N. David Msrniin, 

The course was offered to students via one of two treatnsnts — 
a Iccture-reci tation-laboratory (standard) n:9thod of Instruction and an 
audio-tutorial method^. The content of the instruction was very similar 
in the two treafcin-nts. In addition stud=}nts in both treatrrjncs had the 
sar,:? hoira-zork assignments, closely similar laboratory assignnients, and 
identical examinations. The difference in treatiii^nts , therefore, was pri- 
marily one of instructional method, rather than of content, assignments, 
or examinations. 

The standard method of instruction included two hours of lecture 
and two hours of recitation per week, and a two hour lab every other week. 
(Recitation refers to a small group session used for problem solving, dis- 
cussion » and quizzes.) There ware lecture sessions at two scheduled hours 
twice a week, with about 125 students assigned to one of these sessions, 
md about 300 assigned to tlic other session. On the average, 26 psrsotis 
were assigned to each recitation section, and about 12 to each lab section. 
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Tlio lectures wore l.aiu;}ii-. by I'rof. John Si jcox» \;!io also '.•/;>!, Uui to-aulhor 
of r;.ilt;r i.ils ii:.c\l \i\ Vi\?. an.! io- tutorial troo ixionl. Wicilallon'y v,;.'ro uiu:i'it 

^/arc Uught iiy (jrudua t'.- tCtiLhlnci ussistunU. 

The iiiiclio-Luliorlal i;i:?L'nocl -inc:! lidcd one liour of rccitatior! pc?r v;joJc 
This hour of group in5truc;l;u)n v.'as 1nc1iic!c;cl prii^arily to provlcis for group 
interactioM, as wall as student contact with one particular instructor and 
an opportunity Por testing. In the audio-tutorial ML'thod; all other initruc 
tion took place at the student's convenience in a learning center v/hich v/as 
staffed by tutors (i.e., physics graduato students and faculty membars) 
fifty-two hours per vraek. Materials available In the learning center in- 
cluded detnonstra tion apparatus for self-demonstrations, laboratory ecjuip- 
r^ent, audio-tape comnentaries, and slides. The taped cor:mentaries and slide 
v;ere coordinated v/ith a study ciuide developed by Dr. OTT end Professor ' 
Sllcox. Tape-sl ido-study guide sequences were used in particular to assist 
students 1n learning to solve physics problems. 

BEST COPY AVAILABLE 



l''"-n •;m:vo"I 'f { i-.i;;] • .'^ /-.I . .,...! -j,: , 'j;., |;v; ...:;■/.;•. , 
roa;:oi:l(„i sc'cfio.r.., sdjcJ.ilocl «•(. vUjhl tUrt-uvDl cUvm. I.cur.v for i;-.^^in-s 
i\/ic;p « v;c'oJ,. SL.Kl 'iiis v;.:r.i assiyn.^! to i. s-otion M: a ()ivan cia-is hour 
on the b'lisis of Ihoir othv.v schiululcd c'lasr/^i. c^rul }.:>rsonal preferenc.-. 
Fifto&n or the twenty-two rccituLion soctioni; scheduled ul four oT t!ie 
eiciht class hours v/ero ';elocL(id for incluiioii in Ih- sai'Jy. T!,:.':.- four 
class hours were seloctod bjicausc thrae or More sections v.ere scheduled 
at each of these hours. 

In order to study the effects of student preference for each * ' 
of th3 methods of instruction, v/e used two procedures of assignirent to 
treatments. At two of the four class hours included in the study, students 
were randomly assigned to treatments and to recitation sections wiLhin the 
treatments. At the other tv;o hours, students v/ara assigned to treatfr.ents 
on the basis of their preferences as expressed on a queshicnnaire ad.iiin- 
istered at class meetings during the first v/oek of the semester. Within 
each hour, students v/ero then rando^nly assigned to specific soctio.os 
within the preferred treatm-nt.* Table 1 illustrates the distribution of 
trcatiTents, procedures of assignniant, and instructors aiTiong the fifteen 
recitation sections. 



^ At each of these hours. if:ore students chose the standard method of instrur- " 
tion than could be accom.Todated if section sixes were to be balencod. Therefore 
son^e students who indicated that their preference for the stand-jrd irethod v-'as 
no L strong were assigned to the audio-tutorial sections. In all ther- were 24 
students reassigned in this manner, an average of 8 to each audio-tutorial 
preforonco section. Subsequently six of these students joined standard 
sections, fhese six students were not included in the data analysis 
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s^v-'-jnU "In iiii;:,' s'.:cl*i v'.ii c t..i'j';i'i !: b.y mi? ^ t.,:i:!^!C'! 'U:.i:I. D:;t.:: ; only 
303 slu;:.Miui. in u'.j '.vLi.ilLri U :•-!■. .ki--: IgS-in the standard 

course or trans f'yrrecl to oth-^r socitlon^i of th.:? couriic; 11 lu'.d i ncoriipl ste 
(laid. 
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Im order (.0 suj.];/ i.rc:! l-lrfj.il.-jni. iiK-iMctions withu-it convoi.'iv!- 

or craits prior to or c.l lii:? v.'ry botjiir-rina or L!|.? tr.'.i;:r;:;.S'iu; . In this 
r.tudy, ^.rui;.^ w^n-v (l) }.:;,;)r^:'d ir- stL::'jf:U; on qirj^Uo/rvrir:,', or {?.) 
n:2asurocl by iri'fins 0/ a mth pr-lost of our d25ir;.'VS all coi;-/ic't''vl it th:? 
first iii-2^tirtg of the* r-c:1talion st'ctions. Kroi.'. the available inror.n?rtion» 
thn choicQ or traits to bo tested Tor inloraclior.s v/ith trc;..- L-anU v/di 
based on the roll owing criteria: 

(1) importance of ths trait with respact to tha course end to 
the differences in the treatn&nts; 

(2) reliability and validity of the r.aasureirent of the trait; 

(3) range of responses to the trait measure. 

Of the possible traits determined by the qLiastionnairs and the 
pretest, tha follov/ing three met the criteria: 

(a) niathe;n?.tiC'al aptituch, as rpaasursd by the riathainatics porr.ion 
of the College EntrancD Ey.ciaii nation Bo::rd Schol-j^stic Aptitude? Test (SATiI); 

(b) verbol aptitude, as rreai^uv-cd by the Verb;; I portion of th- 
Scholastic /\ptitude Test (SATV)-^-^, 

(c) mathematics achioveir?nt, i\s noasirred by the math pretest. 



^ The nine-item multiple choice math pretast is a shortened version of a 
twenty-item pretest used in the safr.e course during tha Spring sfr-^si-^r 1^72 
Ihe latter v/as shortened by eliminating items that did not discriminate aironq 
dn-ferent levels of mathefnatical achieverrent, i.e, those iten.s whic'i w^r-^ ' 
answered by almost all or almost none of the studsnts. In contrast to'sATM 
Items, protest items were selected because they «;ere representative of th=» 
inathematical skills required of the students in this specific course. Within 
each treatrrent, the correlation between math pretest scores and SATM was eaual 
to 33, (Please see Table 4.) Thus the portion of the variance in ir.-ith ore- 
test scores accounted for by SATM levels is only 117., and the SATM and j>th 
pretest appear to have masured different student traits. 



Both SATM and SATV were coded in 13 Icvals. 
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{!«r.i1ysis cho ro'kitiori or 1.;k> l!ir:v; tr^iti to the cou.-se Tiri^)"! (jr.-'c.'.-j 
iiriiC'.ilod li);u S.VfVv/fi. i;o|: sicjni l icfiilly r'^lai.ciu lo i ir.it] ru fic!-:? in •.•it'"!:)r 
troatront. "Inns SAfM tind liicttli i)rc't:'3£ scowls v;cv.-o •lOii.'ict to be the bc-st 
predictors or final grade in each Irealr.ent. Th:? st:bS2quent enalys'Is 
considars only \ha'~><i tv/o traits. 



1*0 



Course Adiic'/fcuu^xl — - 



cjsnerally one Ciuix cvory olhpr vM*f.!k, as v/sll as two intorii:; exa-Tti na- 
tions and a finnl exam. Tcstiii-j t.hu<; v;-Jts not or. a sel v-p.u:ccl or c. 
mastery basis in e'ithsr trcatn^nt. Quizzes v/ere constriictod by th2 
senior author (Dr. Ott) , and exams v.'are constructed by ccnimitteiiS 
of rocltatlon, laboratory, and lecture Instructors. 

At the end of the semester a course grade was calculated for 
each student. The grade was determined as follows: 

Each student received 

(a) 0-100 points for lab (based mainly on lab reports); 

(b) O-TOO points for recitation (based rainly on quizzes); 

(c) 0-100 points for the average of the tv/o intsrim examinations; 

(d) 0-100 points for the final examination.* 

The total scores thus could range fro:n 0 to 400. Crade5 v;ere allocated in 
such a way that the median course ^rade v:as C+, equivalent to 235 to 2^9 
total points. 

The course final (jrade therefore reflects achievement on lab 
reports, quizzes and exans, and perfornsance in lab and recitation sections. 
As such, it Is a broad-gduge measure of achievement In the course. For 
this reason tha course grade was used as the measure of achieveirent (de- 
pendent variable) In this study. 

* The variance's of these four subscores were as follov/s: (a) 52; (b) 315; 
(c) 339r (d) 311. Thus lab subscores had less effect than the other sub. 
scores on stud-nt rankings and final course grades. 
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Co.Mp.-ifi'JorJs of laiLlai trait i,-.:_':i-v.!r--^> ('::!•'! or riiial cjrades 
yieldad th'j results given in T<ible 3. Tha cli rreroncas in traits uhich 
sro significant at ,05 aro tfio vollo-winij: 

(1) Uitfiin th2 runclom as5itjpjn?nt snctions, the variance of 
SATM levals is significantly larger In tha audio-tutorial troatnant 
than in the standard treatr.ent. 

(2) For all sections, the variance of SATi'i levels is signi- 
ficantly larger tn the audio-tutorial treatment than in the standard 
treatment. The mean value of uiath pretest scores is significantly 
larger in the audio-tutorial treatr.ent than in the standard treatnant. 

There are no significant differences in the means or variances 
of final grades- 
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tvicljiict; of a;i inLerciction bcjiivi-'isn !:'i£i trait;; (ivrilh pi\^tr2iic 
scor»;» <!nd SAl.'l Tcvel) tind cha tr:u'u-.i.:„Tits Is shov/r» by tha cli vf-jroncoi 
between t!ia rcyrcsriio:: coc fvlciciiLs in thu two troal.Ti^nl*;. The roci're-isioi'. 
equations for the tv/o troatininits iuo as foTlov/s (X is IhG scor£: or. Vm 
ni.rth pretest, Y is ttK^ loval on SAIM, 2 and U are estlnvated graJas in tii.:; 
audio-tutorial and standard treatirants , respectivsly): 

Audio-tutorial: Z = -.5175 + .2533 + 9.1667 
Standard: U ^ -.8495 ^ .4081 Y2 + 10.5352 

IJote that tho coefficients of and of Xg are negative because grades ava 
coded 1 to 13 from high to low and pretests are coded 1 to 9 from low to 
high. 

In order to deterinine whether there are levels on either or 

both of the traits v/hich predict greater achlcveireni' in either treatrnent 

(and thus to determine v/hether the Interaction is significant), we 

analyzed the interaction ur>ino the Johnson and flayman technique--. Ife used 

the notation given by Johnson and fJeynan'^, and in an instructive article 

5 

by Koenker and Hansen . The Johnson -Meyinan technique allowed us to deter- 
ir.ine the conjoint math pretest scores and SATM levels for which it was 
reasonable to assume that the difference in grades between treati^^ents was 
real. The basic statistics are listed in Table 4. 



* Note that if one were to obtain significantly different coefficients for 
a single trait, one v;ould have evidence of an interaction between that trait 
and the treatments. In that case 1t v/ould not be necessary to use the 
Jolmson-Meyman technique to determine regions of significant differences. 
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i.'o sc't the love! of sicjin'flcfiticc' at .10, and tlr^i in&d 

F 

In this cpso V/ - {303 - C)/ 2.7^ - 103.33. l?e obtain-d cvn 
observad valu2 of the test statistic v/^,^„ equal to 7/^.85. Thus sinco 
^"'obs^ •^^» rc^gion of sign! ficcnco exists at tho ten parcant level. 
The region In ths x', y' plane is hyparbolic and consists of two parts, 
as indicated In Figure 1. At the .10 level of significance, the standard 
treatment is preferable to the audio-tutorial treatrrent in terms of ex- 
pected achieve.Tsnt for studants with math pret<2St scores equal to 9 and 
SATM greater than or equal to 725. The audio-tutorial treatrrent is pre- 
ferable to the standard treatment for studsnts with nath pretest scores 
less than or equal to 4 and SATM less than 625. Both ir£;thods of instruc- 
tion are equivalent in torir.s of final grades for studants whose scores do 
not fall within these limits. 
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Thr. foreiioinc! titKi'lysio luts inciictiiocl an 1ni.*i !'<■.?. ti on h:'t'.;.^:tn t'.vo 
traits (scorfis 0!i v. mlh pretest, unci levels on SATM) f.jvJ tlifj in^Uiocl or 
instruction, v;!ion the* ciorjcjndeiU. variabla ir, coui*so cjrcc'e. A sliir.i rict'..^.t 
qu2St1on regains, howevsr. What cispocts of t'no diffornncGS bab/aan tr?>?.t- 
mants account for the interaction? 

• As we have Indicated, the tv/o treatrr.ants used different instruc- 
tional cornpon9n"*'.s. The standard method included t:!Q hours of lecture and 
two hours of recitation per v/2el< and a tv;o hour lab evary second v/eek. The 
audio- tutorial n'.athod included one hour of recitation per v/aek in addition 
to learning center activities. 

Thus, there v/aro major differences betv/een the treatrrsnts in in- 
structional components and in the amount of time allocated to each instruc- 
tional cofrponent. These differences include: 

(1) An emphasis on lecture (larga group instruction) in t'ne 
standard method of instruction; an eriphasis on learning center activities 
(one-to-one instruction) in the audio- tutorial .'".ethod. 

(2) A greater ainount of recitation tir.e (small group instruction) 
in the standard irathod. 

(3) Group laboratory activities in the standard nethod; individual 
laboratory viork in the audio-tutorial method. 

There ere other differences betv/een the treatirents in addition to 
those related to major instructional components. These differences include: 

(4) A greater degree of Individualization in the audio-tutorial 
treatment, i.e.. 
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(a) stuc'?i5i.s could O'vcuin instruction when thr?y v/is{ujd for 
as lori'^j d i th^:y wisiv-vl; 

(b) stud<^'nt.5 could stop r.nd ropoat Instruction; 

(c) students cou'ld obtain iJiiinydi cite tutorial assistanc*:!, 

(5) A fjroater ainount of activa rathar than passive learning in 
the audio-tutorial treatnent. 

(6) The availability of a v/ider variety of instructional tnadia ■ 
studants in the audio-tutorial treatment. 

(7) The coordination of laboratory activities with other acti- 
vities in the audio-tutorial treatment. 

(8) An emphasis on step-wise solution of problems in the audio- 
tutorial treatment. 

(9) Develop:r,ent of notes in the audio-tutorial study guida 

suitable for review. 

Additionally, there inay have bean differences which are not 
intrinsic to the two nr^thods of instruction. For instance, it is possible 
that in these particular realizations of the treatrants, the trertt.T.onts 
v/ere Interesting to or suitable for students of different abilities or 
levels of achievement. That is, one treatrrent may have been better for 
higher ability or achieverrent students and the other for lower ability 
or achieveinent students. In that case, the interaction night be due to 
characteristics of the particular realizations of the treatR:ants rather 
than of the treatfrants themselves. If this were the case, one v/ould 
expect students who were not well -matched in their instructional rathod 
to stop attending the lecture or learning center while students who v/ere 



ttSI con MlliUill£ ^ 

wsTl natchcd to conMnui' In a* trncl-'.nco. Thii-., for irisLancc , If a <jr>'ittcr 
porCGntago of slutbnt';; froin the lov/ar flbiTily (iATiI) levels st. jfj-i^d attaiKl- 
ir.cj d Itent;. ill L tiuf! SL^i'J.'ii -j/c L!ij hl:];!::r ability (SArr';) Jjvjii.^on^ Mkjhi: 
suspcict ihul Lhif. tnujliiKv.ih v/as losf. suited to ilv-' lower cibilily sludetius. 

In this study, v;.-? found thfii: tliGrc v'erc no niajor di trt^rc-nces in 
continuad cittcjyJarico In K-clure ainong stuclenti h-ivinrj difforonu scorai on 
the SATM. Mov/ever, there was soma tundsncy, thouglj not strong, for students 
with higher scores on the math pretest to attend a greater percentage of 
lectures than those with lower scores on the pretest. There was no relation 
bet^'faen continued learning center attendance and SATM or math pretest scores. 
The observed interaction is, therefore, probably due to differences inherent 
in the treatments, e.g., those labeled (1) to (9) above. 

Of these nine ite»ns, those are of interest which tend to account 
for the fact that correlations between the math pretest and achievement (and 
betv/een SATM and achieveir.ent) are lower in the audio-tutorial section than in 
the standard section. Thus we are interested in those items v;hich tend to 
obviate differences in mathe.natical achieveirent and cbility amonj the audio- 
tutorial students. In particular, we are interested in those differences 
beft/een treatments which (1) give an advantage to high ability students in 
the standard treatment relative to those in the audio-tutorial treatrrsnt; or 
(b) give an advantage to low ability students in the audio-tutorial treatirent 
relative to those in the standard treatment. . / • 
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(?.) m[iht t»-'ivl to luinoMi. ! j;;:.'V .'ibi "I i ty .;fiJ acfiif;./; :';'mL s^.i!cI j;iLs in th'.i 
audio -U; (.or i a I trL'.il!;;eiil. This v.'juU! r^uiiVl: C.u-2 to Vno. (\i(^'yl-.'V 'incii • 
Yifiun'rixeition of tli^ loarniiiij crjrrler and <!Uc!io-tul-.orial lab in coi'pari son 
v;ith lf?c"ture and tUv. stcincK-ird "Iftb. On tiif; othvr \v\\):\, point {?.) v/oulc! r.c:?;r: 
to fdvor slU'Jr'n.'::. in Mi:.' .'Ufindard treatJiiont b.^cau:;G thast? studonts h-.sd L;ice 
as much tiuta devoted to red ifit ion. Hactius-j recitations g-jnerdHy ernp'orisi^o 
pro!)1ei]» solving. It is reasonablG to assume that they inisht Iiave besn p;ost 
bonafictal for the? high ability or achieveiiiant students . If this v/era the 
casa, point (J?) v;ou1d halp to account for the superiority of those students 
in th2 standard trcabnant relative to those in tha other t)'catin2nt. A noa- 
sure of support for this asGUii'ption comas froiD the foot that in the standirrd 
trcat:r;2nt th^'; students I'.aving hicihc^r imth pretest scores attended a highcir 
percent'?.ge of recitations th.^.n those v/ith lov/ar ni?»th pretest scores. Th.us, 
rocil-^itioij FiViy ha*'e been pjrticularly useful for high achievri;;ent Suuder.ti;. 

Oi'-' it^'His (0 threug'i (9), all i.riiiht be particularly helpful for 
those students in the audio-tutorial treatntent with lower aptitude and 
achieveir.ant levels for tlie follov/lng reasons. Itents (4) through (9) indi- 
cate that students In the audio-tutorial treatment could spend the tine re- 
quired on portions of the subject tiiatter which caused thein particular diffi- 
culty. Moreover, students could v/ork in a step-I)y-step fashion, and could 
engage in as much review as they found necessary. On the otiier hand, the 
kind of instructional assistance described in these items vould ba of less 
irnpor lance to the highest ability students in the audio-tutorial treatrr.ent. 
Therefore, these students v/mild not h .vc an advantage relative to si nil ar 
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stud^ints in the standard trcidtrrjiii: on uu Lti:.1s of itoi.ii (1) throujh (Q). 

Mc conclude, thorofoco, ttiat the obsorvcid ifitoructio.i Is pro'sably 
(fl.'J r.o i i)h ■.•)■•'*:!. •:.!u;v,i(:f i .: i. i f::. oT III.'' '..'.;o • L' : ;;i i/j ^ iiwu c v.* ;.' 

more siii taljlo for studcints in our pupulijlion with tlif; hi'jhcst obiVity ;.uid 
ctchiovop'c'nL J cuid thf> olhor iii^.r.v r-uitf;')!-} for Ihc! sludi^nls in our pop-.ila- 
tion v/ith Uki Uwost tioi lity euid adric'va;r.jnt. Pxocan tfmt tho riaitio of 
SATM scores Wcis from ebout iiOO t.o &nO. The graato>^ dagro:? of iiv:li vidua'! ~. 
ization in the audio-tuLorio.l treatn^jnt soens to hcneiiL lower uhl liLy and 
achievement students relative to siFnilsr studants in the standard treatment. 
The greater errount of tiiria allocated to recitation in the standard troat- 
iiient, and ths cri^p'iasis in recitation on problem solviog, may £cca unt for 
the iireater achievarrant oT higher ability and achiova:rint studants in the 
standard trcatrr,snt rolativci to sinnler students in the audio-tutorial treat- 
ment. 

These results are in general congruent with Cronbnch and Snov/'s 
Interpretation of earlier trai t-troatirent interaction ;;ti;d1e5, As sun.nsr- 
ized by Salomon^, treatiiient;; v/hich force students to pay attention to detail 
appear to bc;K?fit lew rion:^ral-abi li cy students, ir. ralacion to hlcjhnr abilit 
students. On ths other hand, treatiTients requiring rapid n.^nipulatioii of 
symbolic rreaning appear to benefit the higher ability students rather than 
lo'// ability students. 

At. present, it is impossible for us to geniralize from t!»e inter- 
action which we have observed in a study of two treatii^ents in one physics 
course to other treatrr.ents and other courses. * It would be n-ost useful in 

* This research is continuing under a grant from the Sloan Foundation. 
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this rcMiar-.l it otli';r inv.'islifjat.arr, v.oi.'lcl revicv; Livr-'ir rcjiit'orch co::;;)v'.rin.i 
poss i!) !o 1 c-t.rocit;! ' r.'u: iiii.;! acti oils .■•'• 
WC' lifive i.;>cd a trtri i.-troal.iLL^nL interocllun npproiic'j to aiidly;?^ 
tiie iichiftvoi.v.'r.l; of coTk?fj3 physics stL!c)?nU in two i ns I; n.'C Lionel Miod:i--- 
an eiiidio-tULOrial nii'thod of instruction and a lecLurc:-roci taticn-laboratory 
liiathod. Vi3 have found indications of an interactian betv/L'sn niatnetnatical 
optituda, iichiavcMrsnt in mathematics, and the insfcructlonal rr.et'nods, Tho 
dc-ph-.y'cint variable v;as tha students' cichiovaincnt in the; course as ir-oasured 
by CO ursG finnl grade?. 

The locturo-raci tation-'IcVooratory tn^'thod appears to ba nioro 
suiu.bla for students with the hi^ii-Jiit mtithai-'utica"! cibiVity iind achicveiiient, 
the? audio-tutorial nocJiod nnro suiLcible for stucL^nts with lower 
ci:!';i:ical cVolVity and cichicivcirerrc. Bolh n;:?thudi of inst^Hiclion arc nquiva- 
l-r^.nt in tiMi.ir. of final grfiJjs for stjd:.:(;tr> bnU'.Jon t!!-'S2 two c;:tr:j;.:\;s . 



* V/e thank Wtiltcr ?ec\ercr and Oasoii MiUriicin for h:!lprul sufirjcstions. 
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or 0;-i>tjon Prosi;, Tuvh-OLs 1900), pp. ?<--3':>, 

2. D. (;. Bfrlin-r cnnd L. S. Ctihen, I'oy;Ipv/ pf_IL^^^^^ 

f>8 (19/3); L. J. Cronbach and R. n. Snov/, "Individual DiffcirencGS 
in Learning Ability as a Function of Instructional Variables" 
(Stanford University), pp. 1-25. 

3. For a general dv?scription of the original fonn of audio-tutorial 

instruction see S. M. Postlelhwdi t, J. D. Movuk, and A. T. Murr?sy, 

» Jil9_A'J-liliIi'i?;ci^QJ!l£i.f^^^ (3rd Gd. Burciass, 

Minnaapolis, 1972). 

4. P. 0. Johnson and J. fiey.'nan, ^iilij.^ij ca 1_ |[esj?aj^ 57 

(1935). 

b, R, II Koanlcer c-nd C. II. Hcinscn, journ-yl of t'xp?nri£nital Education 10, 
16^r (1942). 

6. G. Salomon, £lvlfn;j_j)^^(Lg^i£catlpnal R.^soHrdi 42 , 327 (1972). 
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"iUj joint frtsiu-'iicio-s oT S.MM iinJ prot-i.'. r.cor.--, for oar.fi h-e:-.:> 
arc ii»tr,c«t.fV. Dl wch poinL (>:', V). Yhr.. ivj-I.-r of data prrinU 
tiu- joint nequfincicrO for thn citKlio-tutorvn t.reaLr.i;nl. arc 
listad at tiur uppar ricjht at c-cit point (x'> y'), tind those- for Ui.^ 
standard trcatniCMl arc lisceci at tlio lower Tart at each pr.int (x', y'). 
The SAT?-1 levels are indicated on the; y axis, with thair rcispactive scores 
indicated in parentheses. The inath pretest scorss are indicated on ths 
X axis. Tfie scores of sti!d2nts highest in matheri^aticaT aptitude and 
achievement as li^iidsured hy the two traits are located in the upper ricjiit 
hand corner. The scores of students lowest on both traits are located 
In the Ic'wer left hand corn-r. The 1 ins 9' runninrj^ diagonally from lov;?^r 
lc:ft to upper right is the av.U of the hyperbolic region of significance. 
The o.^ line intersecting th. o>: lir.. is the lino or non--sic,nif icanre. 
On the upper right side of the lin? the value of th?. difference in 
predicted grades (i.e., 7- 1!) lease rurginnlly in favor of scud '.nr.'..; 

•in the standard treatment. On the lower left side of the o .^ line l!v- dif 
ference in predicted grades is in favor of students in the audio- 
tutorial trcaUnent. The portion of the hyperbola in the upper right 
corner is the region in which students in the standard treatirent can be 
expected to achieve higher grades than students in the audio-tutorial 
treatment, at the .10 level of significance. This region includes stu- 
dents having math protest scores of 9 and SAT.M scores of 725 and above. 
The portion of the hyperbola in the lower left corner is the region in 
which students in the audio-tutorial treatroent can I- expected to achieve 
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higher grades than sl«kJ??!i.:-. iti the sUui:!ar;! lrt.-a!:;:('ril, ut the; .10 "li-v 
loss than or equal to tibouL ^, , iinci SMM scovi.'S lo.;:'r Uian C?!.». 
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}"!; cv--n R l-." ^ in,; S::\;livi!'.. 

h>u\iii\x^nt Prc'cwlniv Class ri-L'Mnci K-cl t;: Lien inr.tr\:ct:ori.' 

c> AMdi o-Tu Lon V ] Si;^- ird_ 

R.andoin 9:03 A, P> C, D, E 

Random T^:20 F C, C 

Student Preference 10:10 H, G I, F 

Stud<?nt Preference 11:15 I H, A 

■^Instructors A, C, F, G, H, and I taugiit two sections each. B, H, and I 
are faculty inG.Tibsr-; tha rf'Piatndr-;r arc graduai:e students. 
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Cori elation Co-rriv.ir.nr.s oT 'u wW.-. \-:\\.\\ } ina"} Grario, l.Mthin 
TrciUl'ia.iiS. 

Audlo -Tul'.oria'l Trocttrn-ni. Standard Ti OcVtroGn 
(n =• 110) (n 172) 

.28 .37 

s/vrv .18 .09 

Math Prates t" ^.33 ^ 

*TIi.- negative valua of tf - correlation coeff icient is clue to the fact that 
math pretest scores v/ere coded from low to hic;h, wheroas tha grades were 
coo':?d from hi>'h to lev/. 
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Bctsic. Statistics for Ooimsnn-li^yi^an An;'. lysis'^" 

Audio-tutorial Stciricldi d 

- 115 = 188 

:-leanj; / ^^0.9217 --^ 6.4894 

= 4.8783 " ^•7^i^7 

2 = 6.8595 u = 7.0585 



^x' ^ 1.5736 <Jx" - 1.7717 

'^y' - 2.6095 - 2.1911 

- 3.3679 ^li = 3.6916 



r^, —.3093 r^,_, =^-.48/6 



* X ^ r-hth Prcto^t Score; Y = SATM Invel; 1. - FinrVI Grecle in the. '"ludio 
tutorial trt?'itii:i^nt; II " fl.'i.il Gviic'e In ur? Standard trcatr.'.c'nt. 




Math Prfiiest 
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Slluncc, iaciuJ In;', s i .1 '.nu-.c chirinv; ou ! '/.^u .r; and f iUis 
Coui'Ho and cl.'u;;"irooia ridiiiin IstraLion 

(actually glvins ajsi ^iiraent s, collecLlns bomev/ork, calling clnss to 
order, handlnj; out papers, asking for qinct, dl.scussing grading of 
exar.ir;, etc.) 

Reference's to cditcnt of . assignments, exams, other parts of course 
such as lecture or lab 

(to content of a prior or upcoming exam or homewoiTk assignment; to 
marerial covered in lecture [recitation]; to lab work; to a.t. ma- 
terials; brief reference to topics of upcoming lectures or recita- 
tions [naming the topics]; giving answers to problems without explain- 
ing them.) 

Describing a physical system or process or a demonstration; using a 
diagram or n graph to illustrate a process or kind of motion 

Stating a problem or rephrasing a problem 

Definlni; a concept, stating a theory, presenting a model, pointing out 
analogy 

(definitions may be presented in terms of words or equations) 

Applying mathematical techniques or operations without reference to 
physical implications or meaning 

(performing a natliematical operation; deriving a mathematical concept, 
or formula; correcting a mathematical error) 

Deriving or explaining an equation or formula with explicit reference 
to physical mecining or Implications; discussing units 
(dL.cussion of meaning of a term In an equation or meaning of an equa- 
tion; deriving a formula for a general case or a specific example, v;ith 
explicit reference to the physics involved) 

Oigani/ing or structuring the material 

(transitional stateme.J e.g., saying what will be discussed and why; 
reviews or previews of topics) 

Beginning to specialize to a particular case from a more general treat- 

I'.ient 

= Beginning to generalize from a part....:ular case to a more Inclusive 
treatment 

- Conversation only tangentially related to subject matter 
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Appeiulix C: hohavior Catcnorior, 
0 noiu^ ot the otlu^rw 

A « writing ^^n equation or words on blackboard 

B * writing an cciuntion or words on viin;-graph 

(or sb'owlns such a transparency on view-graph) 

C = drawing a diagra:a on blackboard 

D - drawing a diagram on view-graph 

(or showing such a transparency on view-graph) 

E - erasing board, moving view-graph, or 
removing transparency from view-graph 

F * performing a demonstration or showing a movie 

G = questioning or discussion with one student 

H « questioning or discussion with more than one 
student 

I = presenting a question for use with Instant 
Response System 

J = discussing student responses or correcting 
answer for Instant Response System question 



A;);)i'.i'V! \ lu'.rcuot. ta;>s to (;!>:;prvo 



InMt Mir L \om; for Use : 

Notations aro ukuIg every 30 socomls of the mnubc-r corresponding to the 
two cater.orioR. Durittp, the first 15 seconds of each 30 second interval, 
observe the instruction and decide which category Is represented. During 
the next 15 seconds write duwn the two category number.';. 

Major topics and topic shifts are to be noted as they occur. That is, the 
name of a new topic is to be written down in the line of the topics column 
corresponding to the time at which the topic was introduced. Unusual 
events may also be listed here. Also, names of films, or descriptions of 
demonstrations should be listed here. 

The category labelled "Content" refers to the function of the content be- 
ing presented by the teacher (or by the student, during a time interval in 
which student talk is dominant.) 

Tlie category labelled "Behavior" primarily refers to non-verbal behavior 
but is also used to indicate discussion or questioning behaviors. 

As ar as content categories are concerned: 

~ write down 5, 7. 8, 9, or 10 any time they occur, even if others 
occur at the same time 

othsrwise, write down dominant content category. 

As far as behavior categories are concerned: 

~ write down 1 or J if they occur, even if something else also occurs 
— otherwise, write down the dominant behavior. 
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Appendix E: Observation Record 

u;.»:;. star led at^ 
Obs. ended at 
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Appendix F BEST tOPf XniU^f 

David H, Macklin 

Thvi purpose of thin brief exploration is to select and rt-coramend 
a strategy for the Investigation of selections from tape* recordings 
of recitation sections in a college physics course — the one described 
in the foregoing report. The tapes are from the same recitations rated 
by the observers (p,35). 

The strategy has to meet two initial criteria: (1) It should he oriented 
to teaching behavior, because that is the predominant and important 
activity in the recitations; (2) The dimension or dimensions of teaching 
behavior atcended to should be cognitive ones or related to such, because 
(a) cognitive learning is the primary aim of this course and (b) ^;ariations 
in teaching-learning among the thirteen sections may be expect ad. 

A review of the major recent reviews of studies on observations 
systems related to students' learning was used as the primary method 
of search. The foremost review is Kosenshine and Furst's, in Travers 
(ed.) , Second Handbook of Researc h on Teachi ng (1973). Kosenshine also 
did a review, which was somewhat different, that was published in Weatbury 
and Bellack (eds.). Research into Classroom Processe s (1971). Lastly, 
Nuthall has a major review in a 1970 Monograph of AERA. A partial check 
on coverage was made by using a computer search of the ERIC listings of 
1968 — mid 1974, for college level courses combined with observational 
studies of teaching. These reviews were scrutiniztcd for studieu which wet 
the above criteria and were observations-based and related to student 

t 

achievement. Within the search conducted, it might be noted, almost 
no observational studies of college teaching werT found. 

A few major studies will be summarized jn order to discover the kinds 
of factors found relevant in the relationships of observational data and 
student learning. A study by Furst yielded a complex set of findings, from 
high school students in a A-lesson economics "course". (See Nuthall, 1970, 
pp. 19-21.) The 15 classes Involved yielded three achievement groups, 
relative to one another. The data analyzed were typescripts of the complete 
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wc»re; (1) ".•.the three h ii;h-achicv uii; clauses dit"l-erC'*d from the otherfi 
in having tnorc't-xtcnded' indirect teacher talk, more positive than negative 
immediate feedback to pupil responses^ and more extended participation by the 
pupils.*' (2) A composite variable constructed by Furst, incorporating 
(a) moderate vs. high or low "amount of verbal structuring moves*', "a moderate 
rate oT question-answer exchange", and "a high degree of variety in the kinds 
of lop.ical processes exhibitctd.'* Thc^ high achieving group of classes differed 
significantly from the other ti>;o on this index. Another facet was brought out, 
by Rosenshine in the same AKRA Monograph, to the effect Chat significantly more 
typescript lines were devoted to *'def inlng"and ''interpreting*' ::.n contrast to 
**f act-stat ing*', by the teachers of the hifih achievement classes (Rosenshine, 1970, 
p. 116). 

Two additional findings, based on the Flanders system (1965) of interaction 
analysis, were cited by Nuthall. Tli^y Involve "lecturing" and what is labelled 
the I/D Ratio, standing for *'Inditect*' vs. **Direct" Teacher influence. The nature 
of the components of this ratio derive from summing the subclass entries 
in each of the two general categories: 

Indirect; accepts feelings, praises/encourages, accepts or uses 
ideas of students, asks questions. 

Direct: lecturing, giving directions, criticizing or justifying 
authority* (See Table 1 for the definitions.) 

Furst (as reported in Nuthall, 1970, p. 20) found that the l/D ratio 
was related to class achievement, with the ratio being higher (= more Indirect- 
ness) in the high performance classes. These partic^ilar data point to 
a curvilinear relationship, because the lovf point is the *'average: performance 
classes in association with the lowest I/D ratio. But replications are needed 
to test this, rather than higher poxvered statistical analysis of the one set 
of data. — Regarding lecturing, the lowest proportion occurred in the high 
performance classes (c. 30%), and the highest In the average gro^p (c. 60%) 

Wolfson (1973) reports a study of high school students in chemistry 
and in general science. Teachers in these classes who manifested greater Indi-- 
rectness (Higher I/D ratios) had taught the classes which did significantly 
better on major 5;tandardizod tests. The 5;ame kind of finding was obtained 
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V»V l :tf;iiu»r (j 4*p(M-t fd W ;JnLlri1l, 197(U pp. for U) c:!:ls^;e^^ of 

oi,i^ i;rauv* iiCiui .Ml-.i III M 'i-W:-;'i. htl) 1 i.uni .1 hiol.u.-y lu:*. v*a unit; 

^•roalor iiu! i.r».»ctne8ii vns sj ;,;l\a f Ir/iPt l.y rttlateJ t o ijaiiu; In s^l.udo.ni::^' 
loarning nnU also to ytndiiiil^* positive at:titiulw» Luv^^ard the imit. 

A study by Solumon, IJo/alck, and Roseu;;hlucs reported by Ilosca.shiue in 
Wofttbury and lU-llack (1971, pp. 65-67), also mi-rlLG sonip atrcnnlon. 
Solomon and hir. co--workors coded tape recordln<.';5s , being the obSciirvaMon^, 
to find difEtorences v/hieh wtri! related to two kinds of differences in 
student acli LeveuieuL : {\ain in factual knowled;.^e and gain in cOKiprohension 
"of subject-matter. Twenty-four clas.ses of a college level night school, 
one-- "semester course in Atnciricaa government were the study* s population. 
Two sessions of each cla:,s were rccordedp From these recordings, values 
for 61 measures or dimensions were taken. These were combined with 
100 measures derived froni ratings by observerr. and pupils. (It's reasonable 
to assuiae that rating i^ a sui!iriii?ry depiction, in contrast to a specific 
t lire-and-place observation.) The 161 measures were factor analyzed Into 
6 factors, then related to the two types of gain: factual and comprehenrrion. 
Four of the six factors yielded significant correlations with gain, but 
no factor was significantly related to both types of gain* 

The results may be presented as follows, which shows both the factor 
label the authors used and the measure derived from the observations 
which was most closely related to that factor—in order to give a raore con- 
crete meaning to label. 

Factor related to gain in factual knowledge: 

"Clarity vs. Obscurity" (proportion of student requests for inter-- 
pretation to tota^ student speech [factor . -^ading: -.I>6]) — corelation 
(r) with factual gain ^ .59 (p<.05; linear relationship); r-.04 
with comprehension gain (not significant). 

Factors related to gain in comprehension: 

"Permissiveness vs. Control" (proportion of teacher speech to 
total classroom speech [factor .loading: -.92]) — correlation 
was significant (though not reported numerically in Nuthall's review), but 
the relationship was curvilinear, i.e*, middle levels of "Permissivenass 
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Viu Control" v^m^^ ;ir;:u>c ialed villi hi^;!u!r coinprolK^nslou gain-ri^ Tin: 
relationship to factual yaia waij jn.sigaitieant . 

"Knorj;y vs. LotltnL-};y** (laLlo of teacher requests interpretation 
to student rcquesLts [factor loading: +.70]); r with 
conipi elleu^;ion f,'i^'n (j>^./,05; linear); r ".23 v^iLh factual \\^iin (not 
ttignif: icant) • 

"Flamboyance vs. Dryness" (ratio of teachar per clonal references to 
student personal references [factor loadin{;: ".51[); r *=.A2 with 
comprehension gain (p/..05; linear); r «.08 v/lth factual gain 
(not signif Icmt) . 

In sum, the two things that stand out in the study by Solomon Ot are 
that certain qualities are related to achievenent, anu that achievement 
needs to be considered in a differentiated, rathtvr th/ui in a global in.inner. 

There is a small Get of related stud ies v/hich report invc-^i^t if;at icns 
of "t'xplainini^, ability". XI; is topic or characteristic is more microscopic 
than the dependent variables in the researches of Furst and of Solomon 
e^t al. Rusenshine's study will be presented here, In particular. 

The data for this set of studios come from a s Ingle "source , and arc a 
peruliarly small sample of teaching behavior — althuu^;h there is tiie virtue 
of standardization of "v;hat" was tauijhl. Forty-three teachers in 
high school social studies tauj^ht two 15*minute lessons on contemporary 
events in Yugoslavia and Thailand. Inter-class ability differences x>rere 
adjusted via students' performance on a test following presentation of 
15 minutes of recorded teaching on a third country* Relevant: tests were 
administered following th,r? Yugo.slavla and Thailand lessons. 

Rosenshiufj (reported in Nuthall, 1970, pp* 25-26) investigated a 
large number of measures of teacher behavior, in relation to two sub-- 
groups of teacher^5 — the*- most and least effective, as measured by their 
classes' performances. The three variable:; t\\Pt v;ere significantly rolaced 
to effectiveness (class pi»r Formance, adjusted for level on the ' test*V 1 esson^ 
follow: 

Gesture and >{ove:nent; the more able teachers showed a greater 
tendency to move around tlie room and gC!Sture witli hands, head, and trunk 
Rule and Examj^lc Patterns: the ri'ore able teachers showed a 
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gCeaLer U*iidtuu:y Lt» sf ^lt: rules bclU lu ioro ami :Wlt^r dif;cu;>:;i?i:/, o:viiai>jA'. 
whilv* loss nbUr f suiH!t^t^firiil]' teachers I f.»nUocl C^i 'fXa the* tulc only oaot?; 
ciLhcr Ixjloro or aiici ilic oxaaiplo. 

KxplalnlnK Lhik.s: Lhe muro able Luai^lu.rs tt»iided Lo inaUo i;rcMt.cr 
uSLn oC U.nlcln;i wordii i^nch as "bocaust% tlu'Ceioio, in order to, conr^^'juen t 
by m.\'His of, svucl*, elc-'* (NutU:ill, 19/0, 

From the various studies repoiHed, IL seems warranted to draw a few 
concliiJ::ions or {;eaernli/.atlouii. Since Rosenshiac* and Furst (1973) li:^t 
9 coiiiponent:; that they found correlated with student achievement In tht^ 
studii*s they reviewed, the conclusions from this overview will he compared 
with that J, if^t. 

The Kur.it sttidy indicated, for instance, the iuiportance of (a) "entetuled" 
Indirecr teacher talk, (b) more positive than ne^iative feedback to f;tiident 
talK> U ) liore talk by students, (d) moderate "verbal structurjng lioves'' 
and (e) rniderate rate^ of questions and aa.swerii. Many of these nspf.cts 
are similar to the components of the 'rindirfCt" composite catftp/)ry of 
Klander* * rty5;teni of ob5^^'rv'.i tioa , which was found in thia Furst J^tudy to 
be positively related to achievement. Wolfson'iJ and La Shlor's studies 
supported thc^ .iij^^nif icanca of these kJuds of behaviors alr:o, inde::ed by 
the 1/1) ratio. They, and like behaviors and classroc^m :fi)!eracMc>n p i( terns, 
v/ere ab'O found important in Solomon el^ /vl. ' s study: Moderate 
■'permJjJiJiveness vs. control," "energy vs, lethargy," and flauiboyance 
vs. dryness." Lastly, Hoseushiue's identification of physical '^;esture 
and movement" may (or may not) betoken correlated pwycholo^';ical and 
interpersonal flexibility, in conLra:>t to rigidity. 

The descriptive names Rosenshlne and Furst used for variables related 
to the above are "Varia1>l 1 ity ," "Enthusiasm," "Criticism" (typically, a 
negative relationship to student achievement found), "Teacher Indirectner^s," 
and "Use of st ructurlnj^ comments" (1973, pp.]56"7)* 

The second theme that can be foiuid In the conditions of studrnt achieve 
meat, from the above studies, is much more specifically "co^;n i t ive ," in for-' 
mat tonal, or rational/mentaT In nature. 
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FitrsL olt.;oH, as promotive of student UMruin,;, t:oachc*rri exhibici.iis n 
variory of loj-.ioal pruc't,Mvsv\-; , ;micI lc*is 'Meoliiriiij;/' Roiicashinii' s coniiiieuf.s 
on Furst'f? Rdidy added tlvit thoru wtsre more Incidents of definLug and 
inturpretiuj^ in the hli^ih-achievLnii classtjs, comparod Co fact-stating. 
Si>lnmun et .aj.'« study indicated "clarity" wn« important for gainina 
lactual knovN^ledi^a, but it vvas not related to comprchenoion. Then, Roseniidune 
study of micro-units of toachini; behavior indicated tbiit both explalnin;; 
links (''therefore/' "since/' etc*) and rule*-t xaiuplii-rul^ patterns 
were impoctaut . 

Rosenrthine and Furst, in their review of the correlational Ktudies 
(1973, pp. 156*7) give "clarity" and "multiple levels of questions or 
ct!gnitive discoui^e" as the catoj;orles that are related to the fore-going. 
— Thus, "task-oriented and/or businesslike" is the only one of their 
nine variables that is not f .talleled in the above net. Ilcnco, the 
generil paralleli> betvc*iath.s '•eview and the suiv/iiary list from Rosenshine 
and Furst support the ImpMr it . argument of thi ^ reviow — that these 
types cu variables are the ooes of appatcnt importance and therefore 
should take precedence. 

The foroHolnj; studies sui^fjest companion c : rcunistanccs *as being 
important. First, interpc^rsonally responsive interatalons between teacher 
and students ( and students and students), such as "moderate" control/ permis^ 
siveness, nu^derates rates of questioning and answering, and of lecturing, 
'more extended" student p-irt Ici patioa (rather than "rapW-f ire" exchanges), 
etc. have been ^^hown to be significant. These- seem to ho highly 
compitible with thi-' indi rect aHpects of tho* Flanderc nystcm of classlf yinj; 
teacher influence. (See Tnbl^^ 1 of this apijndix.) 

The indirect inI:.Uienc»»s would reasonably appear to incorporate at 
loas^. two diniensions of learning environments: (1) an atr!o:,phi»re of 
warmth, more egali t arianisn (thoitah not pnpM control), Interpersonal 
respect; these arc the value aims of Flanders, in fact. (2) more 
opportunity for student participation, for instance, In terras of f?ome 
initiative in asking quentlona, and practice (overtly and covertly) in 
using ideaa, thinking logically. CouKoquently, the FJan.lerB cla.-.sif Ication 
nyKten i£4 recomendyd for in(*lu:^;ion in studf;>s of phyf:ic.s couraeii or any 
college course. 

3G 



Importranci^ of cof ;ni tivc:?, coiupunenls of leacliinij and Iciarnlng. Variety 

of 10{;lcaX processes, more iivitances of teachers' defining and interpreting 

vs. fact--stating, the utility of •'explaining links" and rulG-cxainpio-rulc 

patterns are indicators of this broad class of phenomena. The sytitoni that 

has been spcciCically desijjned Lo differentiate this doaiain is the* 

Bloom taxonomy of cognitive behavior (1956) • Its basic premise is that 

there arc levels of cognitive activities that differ in complexity and 

that can be arranged in a hierarchical order with respect to complexity 

and inclusiveness^ The simpler levels are factual, then "translation,*' 

moving into interpretation, applications of knowledge, analysis, to the 

luost complex, namely synthesis, and evaluation. Bro^^m and his co-worUors 

at Florida (Brown £t ^al, 1971 and Webb, 1970) have. refined and further 

systematized Bloom* s work, for readier application to teaching events. 

Thus, this Florida system is recommended here, (See Table 2 of this appendix). 

Additionally, a serious attempt should bo made to exanine and 
classify the cognitive It^vels and their relative weightings which are 
minlinally necessary to meet the achievement performance criteria of the 
course, most centrally the examinations. Not only ha*^ research shown that 
dilferent conditions of teaching/ learning are related to different typos 
of criterion parformancc (cf. Solomon ct_ al. ) , but it cannot be taken for 
granted that differentiated classroom cognitive tasks, in the Bloom f^-^nse, 
are appropriately reflected in the things the students "really** have to 
learn, namely what is required of them in order to earn their grades. 

Rosenshlne and Furst point out (1973, p. 158) that, where tested, 
such findings as have been presented here, based on correlational studies, 
have not been replicated in experimental studies, i.e., those using explicit 
manipulation of teaching behaviors as independent variables. One view of 
this non-replication v/ould be that the variables manipulated arc not the 
critical ones . A different view would be that sQmething(s) critical is 
changed, but unmeasured — in both types of studies — such that an 
experimentally designed course is not comparable to the 'natural' ones which 
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have beeii observed* Ailhtmfih a cotilcuii: at^.l.Ly;>L^i ol tin*i phynlc;; courL;^' 
would coat>tit:ute a c*i>r r^* l.at:ional study, KtudiCvS of aay college courses actt 
too scarce, Thii^rufore it y.o.cmi^ vortti thci effort to undortahe a study cT 
thi:i phyyicG course with respect to the cognitive and interactive* varl^ibles 
roco:i;niertded • A nuch more anibiticuir*, presumably cjcperlinvn tal ^ proj!;ra:u 
of ytudici; would be to^quirod to invvvstigate the Hourcos of the di.screpancies 
in fiudini;.s of tho ivv^o lype.^ of studios. 
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Interaction Aualyuis in its Uogular Tcn-C.i tc^yory Form^- 

Teacher Talk 

Ind irect Inf lug aca 

1. Accepts Feeling: accepts and clarifies the feeling tone of the 
students in a non-thrcatcainn manner^ Feelings r.iay bci positive 
or nsigative. Predicting or r(:*calling ft?elings are includfid* 

2. Praises or Kncourage,s; prairies or encourages situdent action or 
behavior. Jokes that release tension, not at the eicpeuse of 
another individual, nodding head or saying '*um W or *'s<^ on*' 
are included. 

1. Accepts or Uses Ideas of Student: clarifying, building, or 

developing ideas or suggestions by a student. As teacher brings 
more of his ideas into play, shift to category five. 

A. Asks Question: asking a question about content or procedure 
with the intent that a student answer. 

Di rect I n finance 

5. Lecturing: giving facts or opinions about content or proct-dure; 
expressing his o\m ideas ^ asking rhetorical questions. 

6. Giving Direction: directions, commands, or orders to which a 
student is expected to comply. 

7. Criticizing or Justifying Authority: statements intended to 
chari^'.e student b^^havior from non-nccopt [ahl e] to acceptable 
pattern; bawling someone out; stating why the ttMcher is doing; 
extreme self-reference. 

Student Tal k 

8. Student Talk — Response: talk by students in response to teacher. 
Teacher initiates the contact or solicits student statement. 

9. Student Talk — Initiation: talk by students which they initiate. 
If ''calling on" student is only to Indicate who may talk next, 
observer must decide whether student wanted to talk. If he did, 
use this category. 

(Neither of ab ovej 

10. Silence or Confusion: pauses, short periods of silence and per- 
iods of confusion in v/hich cornmunication cannot be understood 
by the observer. 



* From Flanders, N.A., T eacher Influ enc e^ Pupil Attitudes, and A chieve- 
ment. Coop. Res. Monogr. No. 12, OE 25040. Uashiugton: U.S. Dept. of 
H.E.VJ., 1965. [In turn, taken from p. 35 of Ober, U.L., The recipro- 
cal category system, J. Res, d Devel. tn Kdu c, 1970, 4(1), 34**50.] 
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1.10 Knowledati of Specif ic:i 

1. reads 

2. spL^lXs 

3. itlcntifif^s {jon^^thlag by nauio 

4. dotlnes iiii^anin^i oC term 

5. slve.s a specLflc fact 

6. tolls about an event 

1.20 Knwltid;;e of Ways; and Moans of Dealing with Specifics 

7. recoj;nizes symbol 

8. cites rule 

9. gives chronological sequatice 

10. gives steps of process, describejs method 

n . cites trend 

12. names classification wystem or standard 

13. naTnes what fits i;iven systc-m or standard 

1.30 Knowledge of Unlversals and Abstractions 

lA. slates penaralii:Cid concept or idea 

15. statrrs a principle, law, theory 

16. tella about organi::uiLion or structure 

17. recalls name of principle, law, theory 

2.00 Translation 

18. restates in own words 

.19. gives concrete example of an abstract idea 

20. verbalizes from, a graphic presentation 

21. translation of verbali7.ation into graphic form 

22. translates figurative statements to literal statements 

or vice versa 

23. translates foreign language to English or vice versa 
3.00 Interpretation 

24. gives reasons (tells why) 

25. shows similarities, differences 

26. summarizes or concludes from observation of evidence 

27. shox>?s cause and effect relationship 

28. gives analogy, simile, metaphor 

29. performs a directed task or process 

A. 00 Application 

30. applies previous learning to a new situation 

31. applies principle to new situation 

(continued) 

TVrom Brovm, Kob Burton et al., Pj^ nprUl^axcinovx^ 
B ehavior , 1971. 
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Tablo 2 (contM) 

'M. :-i;i)lLi^s .ilrstCicL ivnovU-ci;^* I i .i ^u'^irtl: L<.-<il l ;;uat- Loii 

33. iacnririos, fiolv^cts, and carries out prorcrts 

5.00 Analysis 

34. <lit;Linguishv^3 fact from opinion 

35. distinguishes fact from hypothebla 

36. dlst Lnguislies conclusions from statements vhich support 

37. points out unst.itod asiuimption 

3S. shows intoractiou or relationship of eleinent:^ 
39. points out pm ticulars to justify conclusion 
AO. checks hypothesis \;ith given information 
Al. distinguishes relevant from irrclavant statemeats 

42. detects error in thinking 

43. Infers purpose, point of view, thoughts and feelings 

44. recognizes bias or propaganda 

6.00 Synthesis 

45. reorganizes ideas, materials, processes 

46. produces a unique connaunication or divergent :dL3 

47. produces a plan, proposed set of operations 

48. designs an apparatus 

49. designs a structure 

50. devises a scheme for classifying information 

51. formulates hypothesis, intelligent guess 

52. makes deductions from abstract symbols, propositions 

53. draws inductive generalization from specifics 

7.00 Evaluation 

54* evaluates something from evidence? *' 
55. evaluates someth'-ng from criteria 
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